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ABSTRACT 

This study investigated the construct validity and 
population generalizabil ity of the NTE Core Battery using two 
different data bases and several analytical designs. Part I of the 
study used data from the November 1982 administration of the Core 
Battery at the subscore level for 5,183 examinees. Part II used data 
from the October 1985 administration of a revised version of the 
battery at the item level for 13,059 examinees. Both parts of the 
study used confirmatory factor analysis to model the structure of the 
test scores in relation to the knowledge and ability (constructs) 
they purport to measure. In Part I a nine-factor model, one for each 
subtest, was tested, but factors correlated too highly to be 
different constructs. The simplest model found to fit the data was a 
three-factor model of general academic skills, mathematics, and essay 
factors. This model fit the data well and almost equally well for 
white males and females and black females. The black male population 
was small, and results were somewhat unstable. Additional exploration 
of models in Part II resulted in the conclusion that the Core Battery 
measures general academic skills, mathematics, and essay writing. 
Eighteen tables present analysis findings. Three appendixes present 
information on correlations and six additional tables. (SLD) 
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EXECUTIVE SUMMARY 



This study investigated the construct validity and population generalizability of the NTE 
Core Battery using two different data bases and several different analytical designs. This report 
has two purposes: (1) to explain the logic of construct validation in general, using a specific 
construct validation methodology, and (2) to explore the construct validity of the NTE Core 
Battery in particular. 

Part I of the study used data from the November 1982 administration of the Core 
Battery and worked with subscores. Part II used data from the October 1985 administration of 
a revised version of the Battery and worked with data at the item level. 

Both parts of the study used confirmatory factor analysis to model the structure of the 
test scores in relationship to the knowledge and abilities (i.e., the constructs) they purport to 
measure. According to its specifications, the Core Battery measures achievement in three broad 
areas: Communication Skills, General Knowledge, and Professional Knowledge. 
Communication Skills and General Knowledge each consist of four subtests that should be 
somewhat related in content but still be different from one another. Statistically, each of the 
four sets of subtests should be moderately correlated but not so highly correlated that their 
scores are redundant. 

In Part I, we tested a nine-factor model (one factor for each subtest) with data from the 
November 1982 administration. Results indicated that the factors correlated too highly to be 
different constructs. Furthermore, the subtest scores did not group into the three factors 
defined by test specifications. 

We then compared the nine-factor model with various other models. The simplest 
model to fit the data was a three-factor model consisting of general academic skills, 
mathematics, and essay factors. The simpler model fit the data very well, accounting for nearly 
all of the variance explained by the nine-factor one. The general academic skills factor consisted 
of seven subtests: Listening, Reading, Writing (multiple choice), Literature and Fine Arts, 
Science, Social Studies, and Professional Knowledge. The mathematics factor consisted of only 
the mathematics test, and the essay consisted of the two essay ratings. 

Implications for construct validity were that the Battery was only construct valid in the 
sense that three factors -- general academic skills, mathematics, and essay ~ were distinctly 
different from one another, but there was no discriminant validity between the seven individual 
subtests within the general academic skills factor. Furthermore, mathematics was different from 
the other General Knowledge subtests, contrary to design. 

To investigate population generalizability we tested the same model simultaneously 
across four populations: White males, White females, Black males and Black females. While 
the model was somewhat unstable for the small sample of Black males, we found no evidence 
that the test was biased in the assessment of any of these groups. 
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Based on the outcome of Part I, we conducted a second construct validity study using 
individual item data from a more recent, revised version of the Core Battery. In Part H, we 
computed tetrachoric correlations among a sample of 119 multiple-choice items and used these 
correlations for later analyses. A three-factor exploratory factor analysis produced a general 
factor, a mathematics factor, and a factor with a few items from the Social Studies test. 

An eight-factor confirmatory factor analysis produced estimates that suggested that 
mathematics was different from all other subtests, the essay ratings were also different, and the 
remaining subtests correlated rather highly. Social studies showed slightly more uniqueness 
than the other subtests of General Knowledge. Literature/Fine Arts, Science, and Professional 
Knowledge each correlated highly with the three Communication Skills factors. 

We hypothesized a three-factor model consisting of the a priori constructs of 
Communication Skills, General Knowledge, and Professional Knowledge. When we tested this 
model and compared it with a two-factor model consisting of only a general academic skills 
factor and a mathematics factor, we found that the two-factor model fit better and showed 
greater discriminant validity than the three-factor model. 

Several attempts to "control" for the effects of reading ability on test performance were 
unsuccessful. First, the division of the population into high-ability and low-ability groups failed 
because the high-ability readers had such a restricted range of scores that tetrachoric 
correlations were not computable. Second, attempts to modify the models to remove or isolate 
reading ability were unsuccessful because the models could not obtain solutions. Results 
suggested there was not enough unique variance in most items to compute unique factors for 
their respective subtests. 

The conclusion from Part II was that the Core Battery measures three different 
constructs: general academic skills, mathematics, and essay writing. Aside from the fact that the 
Mathematics test and a few Social Studies items are different from Communication Skills, there 
is no other construct validity evident. 
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INTRODUCTION 



Description of the NTE Core Battery 

The NTE Core Battery, first administered in the fall of 1982, consists of three 
components entitled (1) Communication Skills, (2) General Knowledge, and (3) Professional 
Knowledge. Each of these components consists of three or more separately timed half-hour 
subtests. 

Communication Skills contains three multiple-choice subtests and one essay. The 
multiple choice parts are Listening, Reading, and Writing. Listening has 40 items, each with 
four choices; Reading has 30 five-choice items; and Writing has 45 five-choice items. The raw 
score for each subtest consists of the sum of the number of items answered correctly. Two 
raters grade the essay holistically and assign a score from 1 to 6. The final rating is the sum of 
the two scores unless they differ by more than two points, a third rater reads essay, and the 
final score is the sum of the two closest ra tings. 

The Test of General Knowledge contains four subtests, each with five- choice items. 
The subtests are Literature and Fine Arts (35 items), Mathe matics (25 items), Science (30 
items), and Social Studies (30 items). 

The Test of Professional Knowledge is designed to measure the knowledge obtained in 
a teacher education program. It focuses on the processes and the context of teaching. The test 
contains four separately timed multiple-choice subtests, the first three of which are designed to 
be parallel and the fourth of which is experimental. Each subtest consists of 35 five-choice 
items. 

The scores of each multiple-choice subtest consists of the number of items answered 
correctly. In addition, scaled scores are obtained by weighting the raw scores of each subtest to 
produce a composite which is then converted arithmetically to a scale having a predefined 
range of 600 to 690. The actual weights assigned to each subtest score were decided a priori on 
the basis of an agreed upon view of the relative importance of the content components. 

Test results reported to the examinee consist of the three scaled scores plus raw score 
information on each of the subtests, namely, the numbers right, wrong, and omitted. 

The NTE test analysis report contains further information about the tests themselves and 
their psychometric characteristics (Educational Testing Service, 1984). 

Purpose of the Studies 

In the development and production of any test battery, fundamental questions of validity 
and possible test score bias naturally arise. We study these issues not only because the public 
demands proof of test validity, but also because we have to know how to report test scores and 
to advise users in their interpretation. 
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Construct validation is a process by which many types of evidence of validity form a 
single logically consistent network. It tells us not only what abilities a test measures, but what 
abilties it does not measure. Establishing evidence for what a test measures supports its 
convergent validity, a term coined by Campbell and Fiske (1959). Showing what it does not 
measure supports its discriminant validity . Together, convergent and discriminant validity 
provide construct validity . As Cronbach (1971) and many others have pointed out, construct 
validity is fundamental to the proper interpretation of a test score. 

Implicit in any construct validation is an a priori theory or model specifying the 
construct that a test measures and the relationships of that construct with other constructs. By 
a construct we mean an ability, skill, knowledge, or other characteristic that we cannot observe 
directly but must infer from observations. Examples are intelligence, teaching ability, 
socioeconomic status, introversion, and physical fitness. Essentially, a construct is a concept. It 
is an abstraction. We have a concept of intelligence, and our concept goes beyond the ability to 
read passages or solve puzzles. All measures of intelligence are limited and fallible. Because a 
construct is an abstraction and must be inferred from observations, it is essential that we 
determine whether we are drawing those inferences correctly. That determination is the essence 
of a construct validation study. 

The NTE Core Battery was designed to test three broad areas of knowledge: 
communication skills, general knowledge, and professional knowledge. Each of these is a very 
general, multifaceted construct. Within each of these areas, the NTE further refined the 
meaning of the construct with more specific constructs such as listening, reading, and writing. 
Designing test specifications is essentially the task of defining constructs as explicitly as 
possible. It is then the challenge to item writers to create instruments that will measure those 
constructs as accurately and precisely as possible. 

A construct validation of the Core Battery, therefore, requires our demonstrating that 
the tests measure what they claim they measure and not something different. This means, for 
example, that the test labeled "Reading" must measure reading skills, and that the test labeled 
"Social Studies" must measure knowledge of social studies. Further, because these tests have 
different labels, construct validation requires that these constructs be different from one 
another. 

While a test battery may be construct valid for a specific population, it may not be 
equally valid for all populations using it. A test may measure one construct for one group and 
a different construct for another. It may measure the same construct for two groups, but with 
different reliabilities. In each of these instances, the test would be psychometrically biased if it 
were interpreted and used uniformly across all groups. We therefore have to test the construct 
validation model to determine whether it fits equally well for all intended populations. 
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PART I: Study of Construct Validity and Psychometric Bias 

Population and Variables Studied 

Data for Part I came from the first administration of the Core Battery in November 
1982. Only the data from examinees who took all subtests of the Battery were usable. To 
study population generalizability we had to divide the test-taking population into groups 
according to major population characteristics. The most important groupings were by race and 
gender. In addition, we had planned to examine other population breakdowns if the groups 
were large enough to enable us to generate stable statistical estimates. 

While it was essential to analyze the population separately by race and gender, it would 
have been desirable to compare geographic regions as well. We attempted to group examinees 
by region but found that those in the south, regardless of how we defined the regions, far 
outnumbered those in other parts of the country. For example, when we defined the northeast 
as those states above the Mason-Dixon line, there were only two Black males in the northeast. 
After many redefinitions of regions, we concluded that the only meaningful geographic units 
might be three states - Virginia, North Carolina, and Louisiana. Even if we had analyzed the 
data from these states individually, excluding examinees from all other states, the numbers of 
Black males would have been to small to include. We decided that, for this study, we would not 
attempt an analysis by geographic region. 

Another population division that we planned to make was according to highest level of 
education completed. Over half, however, were seniors, and most of the rest had bachelor's 
degrees. Since they were taking the test in November, it was most unlikely there would be any 
difference between the two groups. Examinees with more than a bachelor's degree and those 
who were less than seniors were far to small in number to treat as separate groups for analysis. 

One further variable that we considered as a possible population grouping was 
undergraduate major field. Unfortunately, many examinees omitted this item. Fewer than half 
of the Black males answered it, and only a slightly higher proportion of Black females 
responded. Of those who did respond, the largest proportion of White males had majored in 
general education, while the largest proportion of White females had majored in elementary 
education. Among the Black males who responded, the largest proportion had majored in 
general education and in humanities. The largest proportion of Black females who answered 
the question had majored in elementary education. 

We conducted all analyses by gender and by the two largest racial groups, Black and 
White. The other background questions, because they failed to prove useful for defining 
subpopulations, did not enter further analyses. We included in the study only those who took 
all parts of all three tests and who indicated their ra<''\ gender, and grade-point average (GPA). 
The total number of examinees who took all subtests of the Battery was 6,003. Of these, 9% 
either omitted the gender or race question, or they were neither Black nor White. An 
additional 5 percent omitted G^A. The usable sample consisted of 86.5% of those who took the 
entire Battery. This totalled 5,183 examinees 
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The breakdown of these examinees by race and gender was quite uneven. They 
distributed in the following way: 



White males 
White females 
Black males 
Black females 



791 
3,797 
97 
498 



There were so very few examinees in the other ethnic categories that we excluded their 
data from the study. In fact, the number of Black males was so small that statistics obtained 
for that group may prove to be very unstable. 

Test Performance of Population Subgroups 

The four subpopulations of examinees each performed quite differently on the average; 
the largest differences in means were between Whites and Blacks. Table 1 shows the means 
and standard deviations of each subtest for each group. Scores were usually highest for White 
females and lowest for Black males. For White males, there was a ceiling effect on every 
subtest except Writing, Essay, and Professional Knowledge. By "ceiling effect" we mean that 
the mean score lay within two standard deviations of a perfect score. The average score of 
Blacks, on the other hand, was less than two standard deviations higher than the chance score 
on all but two subtests for males and all but three subtests for females. 

It is important to emphasize that although two or more groups obtain very different 
means on a test, this does not imply that the test is biased for or against any group. If a test is 
biased, one or more groups may or may not score exceedingly low. 

Construct Validation Models - Use of Confirmatory Factor Analysis 

The NTE Core Battery specifies nine possibly different constructs: Listening, Reading, 
Writing (multiple-choice format), Writing (essay format), Literature and Fine Arts, 
Mathematics, Science, Social Studies, and Professional Knowledge. In addition to the nine 
specific constructs, the Battery purports to measure three higher-order constructs: (1) 
Communication Skills, consisting of Listening, Reading, Writing (multiple-choice), and Writing 
(Essay); (2) General Knowledge, consisting of Literature/Fine Arts, Science, Social Studies, and 
Mathematics; and (3) Professional Knowledge. 

The primary goal of this study was to first test whether there are nine different 
constructs and then to test whether the nine subtests group into three general constructs 
corresponding to the areas of Communication Skills, General Knowledge, and Professional 
Knowledge. In most test batteries measuring general academic abilities, aptitudes, or levels of 
achievement, we find only two constructs -• general academic and quantitative skills. It is 
normally only in the more advanced subject areas requiring specialized knowledge that we find 
distinct differences in performance on individual content areas. The study had to answer two 
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construct validity questions: Does the Core Battery successfully distinguish between nine 
specific skills? Does it distinguish between three general skills, in accordance with its design? 

To test whether the Core Battery fit the intended structural models, we used a 
confirmatory factor analysis which, because it does corrections for attenuation, is an 
improvement over the multitrait-multimethod matrix using zero-order correlations as developed 
by Campbell and Fiske. (For an explanation of this methodology, see Werts and Linn, 1970). 
For this analysis, we specify a mathematical model (a factor-analytic one) to represent con- 
structs and their relationships with test scores and with each other. This is a highly appropriate 
mathematical model because it closely parallels true- score theory ~ factors model true scores, 
or constructs, such as listening, reading, and professional knowledge. A factor loading, because 
it is the cor relation between the actual test scores and the factor, models a correlation between 
observed scores and true scores. This number squared is an estimate of the reliability of the 
test scores. Correlations between factors correspond to the correlations between constructs, or 
true scores, and therefore correspond to correlations corrected for attenuation. 

The computer program we used for the confirmatory factor analysis was LISREL VI 
(Jttreskog and S6rbom, 1981 and 1983). We begin with an a priori model that we specify 
according to our concept of what skills the tests measure. From that model, LISREL produces 
a maximum likelihood (MLH) solution simultaneously for all subpopulations. By solving the 
equations for all groups simultaneously, we can compare factor structure, reliabilities, and 
other estimates across populations. For an explanation of the advantages of simultaneous 
confirmatory factor analysis across populations, see Werts, Rock, and Grandy (1979). 

Along with a MLH solution, LISREL produces several assessments of the fit of the 
model. One is a chi-square goodness-of-fit statistic which is somewhat limited because it is 
sensitive to sample size and very sensitive to departures from multivariate normality of the 
observed variables. Large samples and departures from normality tend to increase chi-square 
over what we might expect from specification error in the model. When this occurs, the 
chi-square may be "significant" and mislead us into rejecting a model when, in fact, it fits quite 
well. In other words, small differences between the model and the observed data are 
statistically significant when the sample, sizes are sufficiently large or the distribution of 
variables is skewed. A practical use of the chi-square measure is to compare it to the number 
of degrees of freedom; if it is not much larger than the number of degrees of freedom - no 
more than perhaps twice as large - we can usually conclude that the fit is quite good. 

The LISREL program produces two other measures of overall fit: the goodness-of-fit 
index (GFI) and the root mean square residual (RMR). GFI ranges from zero to one ~ the 
higher the number, the better the fit. GFI is a measure of the relative, amount of variances and 
covariances jointly accounted for by the model. Unlike chi-square, it is independent of the 
sample size and relatively robust against departures from normality. The RMR is a measure of 
the average of the residual variances and covariances. When the normalized RMR is less than 
0.05, we usually conclude that the overall model fits quite well. We can use both statistics, the 
GFI and the RMR, to compare the relative fit of two or more models to the same data. 

The decision of whether to accept or reject a model depends not only on its overall fit 
but also on an inspection of other parameters within the model. There may be a good overall 
fit, for example, but with one of the relationships within the model very poorly determined. 
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Furthermore, when the overall goodness-of-fit is poor, we can often discover what is wrong with 
the model by examining the normalized residuals and/or the modification indices. Where 
these are large we can often infer correlated measurement errors where we have assumed, 
under true-score theory, that errors of measurement are random. This information provides us 
with some direction for modifying or redesigning our model to better fit the data. 

Because the distributions of scores on the Core Battery subtests were often slightly 
skewed, we compared the maximum likelihood solutions with two- stage least squares solutions 
also generated by the LISREL program. In all instances, the estimates were virtually identical. 

For this study, we first tested a nine-factor model because the Core Battery presumably 
has nine content areas. We then tested the three-factor model consisting of Communication 
Skills, General Knowledge, and Professional Knowledge. Finally, we constructed and tested 
several simpler models. Only one model fit the data well and showed clear discriminant validity 
among constructs. We will discuss each model below. 



The Nine-Factor Model 

In the nine-factor model we hypothesized that each subtest measured a different factor. 
In order for the model to be identified (i.e., solvable), we had to have at least two measures of 
each factor. We therefore computed two half-scores for each subtest - one consisting of the 
number of odd- numbered items answered correctly, and the other consisting of the number of 
correct even-numbered items. The use of odd-even scoring to produce part scores can inflate 
the correlations between part scores if a test is highly speeded. The Core Battery, however, is 
not a highly speeded test. The most speeded section is Reading, where 90% of the sample 
complete the entire test within the allowable time limit (Educational Testing Service, 1984). 
For most other sections, more than 95% complete the test on time. 

The Professional Knowledge test already had three parts. It was therefore not 
necessary to produce half-scores for this subtest. Nor was it necessary to rescore the essay in 
any way. The two measures that we allowed to load on the essay factor were the two ratings. 
Although some essays required a third reading, we did not include this third rating in the 
analyses. The Appendix includes the correlations among the half-scores of the subtests, essay 
ratings, and GPA. 

Not surprisingly, the nine-factor model fit the data exceedingly well. The short table 
below shows the three measures of goodness of fit discussed earlier, namely, chi-square, GFI, 
and RMR. 
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Nine-factor Model: Measures of Fit 



Chi-square = 669.84 with 504 degrees of freedom 



Goodness-of-fit 



Root mean square 
residual (RMR) 



Group 



index (GFI) 



White Males 
White Females 
Black Males 
Black Females 



0.979 
0.994 
0.878 
0.975 



0.013 
0.008 
0.042 
0.018 



We expect this model to fit well because it contains nearly half as many factors as 
variables. The only way it could misfit would be if, for example, the odd-numbered Reading 
items measured Writing better than they measure what the even-numbered Reading items 
measure. What the goodness-of-fit indicators do not tell us is whether the Reading items 
measure something distinctly different from the Writing items. To answer this question, we 
begin by examining the correlations among the nine factors, i.e., the estimated true- score 
correlations. 

The NTE Test Analysis (ETS, 1984, p. 72) reports estimated true-score correlations 
among all of the subtests. These figures use KR-20 as the reliability estimate upon which the 
true-score estimates are based. We generated true-score estimates from the factor model. 
Table 2 shows the results from both methods of estimation. For purposes we will discuss 
below, we changed the order of presentation, putting Essay and Mathematics last. No single 
estimate of the reliability of the essay exists. Therefore, we have omitted it from the table. 

The correlations based on KR-20 are very slightly, but consistently, higher than the 
factor-analytic estimates. The conclusions, nevertheless, are the same -- except for the essay 
and mathematics factors, there is a very high correlation between the other constructs. It is 
difficult to justify a statement that Reading is different from Professional Knowledge, for 
example, when their underlying constructs correlate better than 0.9. Likewise, we do not 
expect people's true knowledge of literature and fine arts to correlate nearly 0.9 with their true 
knowledge of science. 

Our next step was to check whether the three-factor model consisting of Communication 
Skills, General Knowledge, and Professional Knowledge would fit the data according to the test 
design. 
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The Three-Factor Model: Communication Skills, General Knowledge, and Professional 
Knowledge 



In accordance with test specifications and with methods of score reporting, the nine 
subtests are expected to fall into three general skill groups: 



Communication Skills: 



General Knowledge: 



Listening 
Reading 

Writing (multiple choice) 
Writing (essay) 

Literature/Fine Arts 
Mathematics 
Science 
Social Studies 



Professional 
Knowledge: 



Prof. Knowledge I 
Prof. Knowledge II 
Prof. Knowledge III 



We therefore tested a three-factor model with this structure. This model fit so poorly 
that no solution was computable. The reasons that the model did not fit are clear from Table 
2. The essay portion of Writing appears to measure something quite different from the other 
Writing subtest and, indeed, different from any other subtest. Likewise, Mathematics does not 
belong with the other subtests called "General Knowledge." We also see from Table 2 that the 
Professional Knowledge factor correlateds very highly with Reading and other subtests. Thus 
the test of this model showed quite emphatically that the subtests do not group into these three 
a priori constructs. 

Because this model did not fit the data, we constructed and tested several other models 
which were reasonable based upon a priori considerations. 



Five-Factor and Four-Factor Models 

When we saw that the Essay did not fit with the other subtests designated as 
Communication Skills, we created a fourth factor entitled Essay. Furthermore, because 
Mathematics was clearly different from the other areas of General Knowledge, we specified yet 
a fifth factor for Mathematics. We then tested this five-factor model: 
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Communication Skills: 

General Knowledge: 

Mathematics: 
Essay: 

Prof. Knowledge: 



Listening 
Reading 

Writing (multiple choice) 

Literature/Fine Arts 

Science 

Social Studies 

Mathematics 

Essay (2 ratings) 

Professional Knowledge (3 parts) 



While the data fit the model quite well, three factors correlated highly with each other, 
and we could not justifiably regard those factors as different constructs. For the entire sample 
combined, the intercorrelations of these three factors were as follows: 





Comm. 
SkiHs 


Gen. 
Knowl 


Prof. 
Knowl 


Comm. Skills 


1.000 


0.943 


0.915 


Gen. Knowl. 


0.943 


1.000 


0.908 


Prof. Knowl. 


0.915 


0.908 


1.000 



Our next step was to simplify the model further and combine Communication Skills with 
General Knowledge in the hope that the correlation with Professional Knowledge might drop 
slightly. We labeled the newly formed factor Verbal Skills. The resulting model had the 
following structure: 



Verbal: 



Mathematics: 
Essay: 

Prof. Know.: 



Listening 
Reading 

Writing (multiple choice) 
Literature/Fine Arts 
Science 
Social Studies 

Mathematics 

Essay (2 ratings) 

Professional Knowledge (3 parts) 
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This model was tested on all four subpopulations simultaneously and found to fit the 
data quite well as we see below: 



Four-factor Model: Measures of Fit 



Chi-square = 3177.37 with 644 degrees of freedom 



Goodness-of-fit 



Root mean square 
residual (RMR) 



Group 



index (GFI) 



White Males 
White Females 
Black Males 
Black Females 



0.918 
0.945 
0.833 
0.942 



0.030 
0.026 
0.055 
0.029 



If we look at the solution, however, we find the correlation between the Professional 
Knowledge and Verbal factors still to be quite high. For each of the subpopulations, the 
correlations are as follows: 



Based on this finding, we concluded that we could not justifiably accept a model 
consisting of more than three factors. 

Exploration of Models with Correlated Errors and Multiple Loadings 

Before testing a simple three-factor model consisting of general academic skills, 
quantitative, and essay writing skills, we tested more complex models. In addition to these three 
factors, we hypothesized that certain pairs of subscores also loaded on unique factors that were 
different from the basic three. 

We allowed the multiple-choice writing scores and the essay ratings, for example, to load 
on a factor called Writing. We set Mathematics and Science scores to load on yet another 
factor. Using LISREL, we test everal variations on these designs and found either the model 
was unidentified (not solvable because there were too many unknowns) or the standardized 
factor loadings on the unique factors were very small ~ usually less than 0.10. 



Group 



r 



White males 
White females 



0.914 
0.908 
0.911 
0.921 



Black males 
Black females 
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Failure to find any model that could justify separating the seven subtests labeled as 
Verbal skills led us to test a simple three-factor model. If that model fit the data, we would 
compare it to the original nine-factor one to see whether the nine-factor model provided an 
improvement over the sim pier one. If so, we would accept the nine-factor model; if not, we 
would have shown that the subtests failed to exhibit discriminant validity and therefore that the 
nine-factor model was not construct valid. 



The Three-Factor Model: Verbal, Essay, and Mathematical Skills 

Because the test scores appeared to cluster into three distinct factors, we next tested a 
three-factor model, naming those factors Verbal, Essay, and Mathematics. Indices of fit are 
shown in the following table: 



Three-factor Model: Measures of Fit 

Chi-square = 4333.69 with 660 degrees of freedom 

Goodness-of-fit Root mean square 

Group index (GFI) residual (RMR) 

White Males . 0.888 0.034 

White Females 0.922 0.030 

Black Males 0.812 0.060 

Black Females 0.926 0.031 



Comparing these estimates with those generated by the nine-factor model, we see that 
the value of chi-square has risen considerably, but that the other two measures of fit are 
excellent. As we mentioned earlier, chi-square is very sensitive to non-normality and to large 
sample sizes, while the GFI and RMR are not. The GFI and RMR both indicate that the 
model fits the data exceedingly well. 

When a simpler model fits the data as well as the three-factor model fits these data, we 
prefer the simpler model over the more complex model because the simpler one is more 
parsimonious. High correlations among factors indicate redundancy among those factors, and 
consequently, lack of parsimony. The three-factor model in this analysis is more parsimonious 
because the correlations among factors are smaller than they are in the models having more 
factors. Table 7 shows the correlations among factors for each population group. 

The three-factor model tells us is that the Essay and the Mathematics subtests measure 
something different from what the other subtests measure. The other subtests - Reading, 
Listening, Writing (multiple choice), Literature and Fine Arts, Science, Social Studies, and 
Professional Knowledge - all appear to measure the same construct, presumably general 
academic skills. 
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The next step is to examine the solution generated by the three-factor model and then 
to see to what extent each subtest included in the common academic skills factor still measures 
something uniquely different from the other subtests included in that factor. Based on the 
goodness-of-fit estimates, we expect there will be very little uniqueness, otherwise the model 
would not fit so well. It is still worthwhile to obtain estimates of that uniqueness to have a 
better understanding of what the subtests measure. 

Table 3 shows the reliability estimates of each subtest based on the three-factor model. 
We compute these estimates from the factor loadings ac cording to a formula derived by Werts, 
Linn, and Joreskog (1974). These reliabilities range between 0.569 and 0.867. The highest ones 
are for Professional Knowledge which, because it totals over 100 items among its three parts, 
we would expect to have the highest reliability. Remember that when we interpret these 
reliabilities, they are in relationship with the general academic skills factor. For example, the 
Writing and Science subtests are equally reliable measures of a general academic skills factor. 
For White males, a general academic skills factor can explain 87% of the observed score 
variance in the Professional Knowledge subtests. 

We can now compare these reliability estimates to those reported in the NTE Test 
Analysis (ETS, 1984) and to those generated by the nine-factor model. Table 4 shows the 
reported reliabilities for the entire examinee population. These estimates are values of KR-20, 
a formula based on the internal consistency of responses to items within a subtest. They are 
usually higher than those generated from the factor-analytic model. Comparing KR-20 with the 
factor-analytic estimates, we find approximately the same rank ordering. Professional 
Knowledge has the highest reliability; Science has the lowest. More informative, however, is 
the comparison of the reliability estimates from the three-factor model with those from the 
nine-factor model. 

Table 5 shows the reliability estimates generated by the nine-factor solution. Because 
the scores used to identify each factor consisted of the sum of the odd and even numbered 
items, the reliability estimates generated in the factor-analytic solution are nearly identical to 
the split-half reliabilities (with Spearman-Brown correction) of each subtest. The difference 
between each of these values and unity is an estimate of the percentage of variance due to 
measurement error. 

We can now estimate the uniqueness of each subtest. The uniqueness is equivalent to 
the amount of reliable variance not explained by the general academic skills factor. It is the 
difference between the two reliability estimates. Tables 6a - 6d show, for each group, the 
breakdown of total variance into three components: the amount explained by general academic 
skills, the amount attributed to random error, and the unique variance explained by whatever 
that subtest measures other than general academic skills. 

We see that for White males, something unique to the Reading subtest explains less 
than 3% of the variance in Reading test scores. This is not surprising because a general 
academic skills factor probably consists primarily of reading skills. Secondarily, reasoning 
abilities, general knowledge, and various other skills are necessary to take any test. We do 
expect the other subtests, however, to exhibit greater uniqueness. For Black females, 
something other than general academic skills accounts for less than 4% of the variance in the 
Professional Knowledge subtests. 
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Looking at the tables overall, we see that the subtest showing the most uniqueness is the 
multiple-choice Writing test. Here the unique variance is about 10%. The higher figure for 
Black males (18.5%) may be valid, but we must question it because of the very small number of 
Black males (only 97). The Listening subtest also contains between 7% and 11% unique 
variance, but these amounts are very small compared to the common variance. If we were to 
find that a math test measured reading more than it measured math, we would certainly be 
reluctant to call it a math test. In actuality, we find that for White males, the variance in scores 
for the Listening subtest breaks down in the following way: 71.2% is general academic skills, 
17.9% is random measurement error, and the remaining 10.9% is the unique content measured 
by the Listening test. It is therefore difficult to justify calling the test a test of listening skills. 
This same argument holds for the other six subtests shown to be primarily tests of general 
academic skills. 



Implications for Construct Validity 

We have focused our analyses so far on the seven subtests showing a large general skills 
component. The other two subtests -- the Essay and the Mathematics test - clearly measure 
something different from all other subtests. Table 7 shows the correlations of the Essay, 
Mathematics, and Verbal factors and undergraduate grade-point average (GPA). For all 
groups, the Verbal and Mathematics factors correlate the most highly. The Essay factor 
correlates only moderately with the other two. It too measures something quite different. 

What these correlations show is that the Essay and Mathematics tests have good 
discriminant validity ~ they do not measure the same skills as the other subtests. All of the 
subtests included in the Verbal factor have poor discriminant validity because they do not 
measure something very different from one another. The seven general academic skills subtests 
combined, however, do measure something different from the other two subtests. 

Discriminant validity is one side of the construct validity coin. The other side is 
convergent validity. We have shown that some tests do not measure what the other tests 
measure, but we have not shown what any of them are measuring. A more thorough construct 
validity study would contain data on specific course grades, student characteristics, and other 
measures of achievement in reading, science, etc. Ideally, it would contain criterion data such 
as success in teaching. 

For this study, the only external data we had was self-reported undergraduate GPA. We 
might expect students' test scores to correlate highly with GPA, especially their scores on 
Professional Knowledge, if that test measures the content of the education curriculum. What 
we found, however, was that Professional Knowledge was behaving as a general academic skills 
test. In fact, its statistics behave as if it were an alternate form of the Reading test. 

What we expect, based on what we know of the education curriculum, is that the 
Verbal factor will correlate most highly with GPA. This is usually the case. The correlations 
between self-reported GPA and the Verbal factor are relatively low, however, especially for 
females of both races. GPA, therefore, has provided us with some small degree of evidence for 
convergent validity, but the evidence is not strong. 
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It is not difficult to understand why the correlation with GPA is lower than expected. 
One reason is that the grades are self-reported, and self- reported grades are usually not 
completely accurate. As a result, they correlate less well with other measures of ability. Not 
only is self-reported GPA less reliable than transcript GPA but GPA may not be a reliable 
measure of how much a student knows. In fact, grades often correlate only moderately with 
test scores. So our findings are not unusual. The only conclusion we can draw is that we have 
not established strong evidence of convergent validity from this study, but future studies 
containing more external measures may do so. 



Generalizability across Populations 

A test is unbiased across populations if it measures the same knowledge, skill, or ability 
for all populations and does so equally well for each. In the language of confirmatory factor 
analysis, we say that the tests are congeneric - that each test measures the same factor for all 
populations. Furthermore, the tests measuring the same factors also have equal loadings for 
all populations. Congeneric tests have perfectly correlated true-scores, hence measure the same 
factor. Multifactor congeneric tests measure the same composite of multiple factors (possibly 
with different weights), hence their true scores may not correlate perfectly. 

We saw that under the three-factor model consisting of a general academic ability factor, 
a mathematics factor, and an essay factor, the model was congeneric across populations. If we 
had found that scores on the Science test, for example, had loaded heavily on the Mathematics 
factor for White males and not for Black females, the Science test would have failed to exhibit 
population generalizability because it would have been measuring a different ability for one 
group than for another. This did not occur. The same three- factor structure fit for all four 
groups. Each subtest measured the same corresponding construct for all groups. We can 
conclude that the Core Battery is unbiased with respect to congenerity. 

Whether each subtest measures what it measures equally well for each group is not 
obvious from inspection alone. From the reliability table we see that the Core Battery may be 
most reliable for White males and least reliable for Black males. To test whether this is the 
case we constrained the factor loadings to be equal for all groups and tested whether this 
constrained model still provided a good fit of the data. 

We would not expect this model to fit quite so well as the unconstrained model because 
the four groups are certainly not identical, but the solution still produced a good fit to the data. 
Indices of fit are shown below: 
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Constrained Model: Measures of Fit 



Goodness-of-fit 



Root mean square 
residual (RMR) 



Group 



index (GFI) 



White Males 
White Females 
Black Males 
Black Females 



0.887 
0.922 
0.805 
0.920 



0.061 
0.031 
0.092 
0.059 



The only group whose data did not fit reasonably well was Black males, and as we 
discussed earlier, the size of this group was only 97. These data alone cannot give us the 
reason for the slightly lower reliability of the test battery for Black males. One condition 
that can yield a low reliability estimate for a group is homogeneity or restriction in range of 
scores. We saw earlier that the Black males had the lowest means on four of the subtests. 
They also had the smallest standard deviations on four subtests. What probably occurred was 
that the reliability estimates were low because only 97 Black males took the exam, they had 
relatively low scores, and they had little variation among those scores. What we have to 
examine in addition to reliabilities are the standard errors of measurement (SEM) of their 
scores. Table 8 shows that the SEMs were not especially high for Black males. If the SEMs 
had been high, we would have to conclude that the test did not measure as well for Black males 
as for the other groups. The SEMs were about the same for Black males as for the other 
groups. It is fair to conclude, based on the small number of Black males, that the test may 
measure the three constructs as well for Black males as for the other groups. 

Referring back to Table 3, we see that the reliabilities of subscores on the Science, 
Social Studies, and Mathematics subtests were somewhat lower for Black females than for other 
groups. On all three of these subtests, Black females had lower than average observed score 
variances and very slightly higher SEMs (Table 8). If there is any lack of population 
generalizability here, it may be that these subtests have a slightly higher SEM. The lower 
variance in scores among Black females may be the cause of the slightly lower reliability in 
those scores. 

White females obtained scores with the lowest reliability on the Essay. Their Essay ra- 
tings also have small variance, probably because their average ratings were high. What is likely 
to be occurring here is a ceiling effect. The White women score so high that the Essay ratings 
fail to discriminate very well among them. Of course, it may be that it is unnecessary to do so. 

The reliability estimates were all quite high for White males. It is interesting to note, 
however, that on the multiple-choice writing subtest and on Professional Knowledge, the scores 
of White males had the highest SEMs. It was only because they had large score variances on 
these two subtests that the SEMs were high as well as the reliabilities. 

What we have seen from this analysis is very little, if any, psychometric bias across 
population groups. No group exhibited consistently larger than average SEMs, nor did any 
particular subtest have a large SEM for a specific population. Overall, the data suggest there is 
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no significant bias in the Core Battery in generalizing score meaning and precision across 
population groups. 

Conclusions from Part I 

The first purpose of the study was to examine convergent and discriminant evidence of 
construct validity in the NTE Core Battery. The second purpose was to investigate the 
generalizability of score meaning and precision across populations. 

We accomplished the first purpose of the study using confirmatory factor analysis to 
model the structure of the test scores in relationship to the constructs they purportedly 
measure. According to test specifications, the Core Battery measures nine different skills or 
knowledge areas, some of which cluster into broader areas, such as "general knowledge." 
Testing this nine- factor model, we found that many factors correlated too highly to be justi 
fiably regarded as different constructs. 

We found instead that a three-factor model fit the data exceedingly well. Seven of the 
subtests - Listening Reading, Writing (multiple-choice format), Literature and Fine Arts, 
Science, Social Studies, and Professional Knowledge - all formed a single factor which we 
labeled general academic skills. None of these subtests, with the possible marginal exception of 
Listening and Social Studies, showed sufficient uniqueness to suggest that it measures 
something different from the others. 

The essay form of the Writing test apparently measures something quite different from 
what the multiple-choice form of Writing measures. Whatever the multiple-choice Writing test 
measures correlates better with what the other multiple-choice subtests measure than it does 
with what the essay measures. This important finding suggests that whether the test is in a 
multiple-choice format or not may be an important factor in performance. While we cannot 
verify this suggestion from the data at hand, we should consider differences in test format when 
attempting to explain the factor structure of the tests. It is possible that the ability to take a 
multiple- choice test, or alternatively, the particular kind of multiple-choice strategy required to 
take the Core Battery is affecting performance on all of the multiple-choice subtests. 

What we found from the three-factor solution was that the essay and the Mathematics 
subtests have good discriminant validity -- they both measure skills that are uniquely different, 
but related, to the skills required by the other subtests. The other seven subtests forming a 
single factor have poor discriminant validity vis-a-vis each other. They all appear to measure a 
common set of skills, probably a combination of reading and reasoning. This is also an 
important finding because, although these subtests are measuring this factor well (with good 
reliabilities), we cannot justify the claim that they measure different skills or knowledge areas. 
For example, uniqueness explains only about 5% of the variance in the Professional Knowledge 
subtests. 

The analysis for population generalizability showed that for the four populations - white 
males, white females, black males, and black females -- the factor structure was the same. This 
means that the subtests measure the same constructs for all four groups. The general academic 
skills factor consisting of seven subtests is the same for all groups. The factor loadings of the 
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subtests on their respective factors were also not greatly different across populations. While 
some of the subtests obtained lower reliabilities for black males, these findings are likely to be 
attributable to the small number (n = 97) of black males taking the test. We can confidentally 
conclude that while the four groups had different means on the various subtests, the tests 
showed evidence of common meaning and equal precision across groups. 
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PART II: Analysis of Construct Validity at the Item Level Using 1985 Data 

Population and Variables Studied 

This analysis used data from the files of all test candidates who took the Core Battery in 
October 1985 (Form 3HNT) and attempted all nine operational sections of the Battery. The 
population size was 13,059. Because we found no evidence of differential score properties 
across populations in the previous study (Part I), we did not analyze the data separately by race, 
gender, or other demographic division. 

We did, however, form two subgroups defined in terms of reading scores. The results of 
Part I had suggested that reading ability may work as a limiting factor in student performance in 
other content areas. If reading skill could be "held constant," perhaps we could detect 
differences in performance on other subtests, particularly among examinees who were the best 
readers. We therefore selected the top and bottom quartiles on the Reading score with the 
intention of conducting the same analyses on those groups as we conducted on the entire 
population. 

The range in Reading scores for the top quartile was from 26 to 30 items answered 
correctly; for the bottom quartile the range was from 1 to 18. Because the distribution of scores 
was so highly skewed, the range was extremely restricted among the top-scoring examinees. This 
restriction in range presented a problem in the analysis, as we shall see later in this report. 

In contrast to the analysis in Part I, the analysis in Part II used individual items instead 
of subscores. Because items are either right or wrong, item scores are binary variables. The 
usual product-moment correlations are appropriate for computing correlations among 
continuous variables (subscores) but are inappropriate for binary variables. An appropriate 
measure of association between binary scores for underlying continuous variables is the 
tetrachoric correlation. TESTFACT is a factor analysis program designed to produce 
tetrachoric correlations and to use those correlations to compute a factor analysis for binary 
data. We used TESTFACT for the first factor analyses and then used the tetrachoric matrix 
produced by TESTFACT as input data for LISREL. 

The entire Core Battery contained 340 items plus two essay ratings (excluding the rating 
used to resolve a large difference between the first two ratings). This was too many variables to 
use in TESTFACT. Furthermore, the distribution of items was such that the three sections of 
Professional Knowledge together contained 105 items-far more than any other section. 
Mathematics, on the other hand, contained only 25 items. This very unequal item-content 
distribution had the potential of affecting the outcome of a factor analysis. 

To equalize the distribution of items across content areas, we sampled every nth item 
from each subtest in such a way as to include approximately equal numbers of items from each. 
The exact items selected were as follows: 
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Subtest 



Sampling 

fraction Items selected 



No. of 
items 



General Knowledge 

Social studies 1/2 1,3,5,7, ... 15 

Mathematics 3/5 1,3,5,6,8,10,11,13, . . 15 

Lit/Fine Arts 2/5 1,3,6,8,11,13,16, ... 14 

Science 1/2 1,3,5,7, ... 15 

Professional Knowledge 

Prof. Know. I 2/5 1,3,6,8,11,13,16, ... 14 

Communication Skills 

Listening 2/5 1,3,6,8,11,13,16, ... 16 

Reading 1/2 1,3,5,7, ... 15 

Writing (mc) 1/3 1,4,7,10,13,16,19,... 15 

Essay Rating 1, Rating 2 



This selection resulted in 119 test items and 2 essay ratings, or a total of 121 variables. 



Correlation Matrices 

The computer program TESTFACT produced tetrachoric correlation matrices for the 
population as a whole and for each of the two subgroups defined in terms of Reading score. 
Appendix B shows that matrix for the population. The matrix was very similar for the low 
Reading-score group, with each correlation being slightly lower for the group than for the whole 
population. The lower correlation would be expected because the distribution of scores covers a 
narrower range. 

The restriction in range of scores for the high Reading group was sufficiently severe to 
prevent the computation of correlations. Recalling that the correlation between subscores is 
quite high (from Part I), by restricting the range on Reading scores we have also restricted the 
range on scores for other subtests. For example, consider the following. Among high Readers, 
on the Social Studies subtest, only 1.6 percent got item 5 incorrect. On Listening Skills, only 0.6 
percent got item 6 incorrect. On multiple- choice writing, only 1.0 percent got item 34 incorrect. 
On the Professional Knowledge test, only 3.0 percent missed the first item. Basically, the test 
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battery was too easy for top-scoring Readers, and the ceiling effect prevented a correlational 
analysis among those examinees. 

Exploratory Factor Analysis Using TESTFACT 

In order to test whether items would factor into the three a priori categories of General 
Knowledge (GK), Professional Knowledge (PK), and Communication Skills (CS), we set up the 
TESTFACT program to produce a three- factor solution. From that solution we computed 
varimax and promax rotations. Table 9 shows the rotated principal factor loadings. 

The large number of positive non-zero loadings on the first factor suggest that the first 
factor is a general ability factor. Some General Knowledge items along with a few items from 
various tests load on the second factor. The only salient loadings on the third factor are 
Mathematics items. No other items - not even the Science items ~ load on the third factor. 

Because the tetrachoric correlations could not be computed for the high reading group, 
the factor analysis could, not be computed either. For the low reading group, results were 
nearly the same as they were for the population as a whole. The only difference was that each 
factor loading was somewhat lower. 

The implication of these analysis is that the Battery as a whole seems to measure two 
factors: mathematics and another general ability factor, mostly defined by verbal items. To 
understand the factor structure more clearly, we conducted confirmatory factor analyses using 
LISREL to test the fit of several a priori models. 



Inclusion of the Essay Ratings 

The 119 multiple-choice items in the test battery were all binary, and therefore it was 
appropriate to compute a tetrachoric correlation matrix. The essay ratings, on the other hand, 
were on a continuous scale. While we did not know what effect it might have in subsequent 
analyses, we decided to compute biserial correlations between the essay ratings and each of the 
other variables and to insert those values in the larger matrix. The essay ratings were labeled 
as variables 120 and 121 in the matrix and in the list of variables that appear in the LISREL 
analyses. Appendix C shows these correlations between the multiple-choice items and the essay 
ratings. 

The Eight-Factor Model 

In the eight-factor model we hypothesized that each subtest measured a different factor. 
This model was nearly identical with the nine-factor model in Part I, except that it allowed the 
essay ratings to load on all of the other eight factors as extension variables instead of defining 
the Essay as its own factor. We did this in case there was an incompatibility between the 
biserial and tetrachoric correlations, anticipating that the incompatibility might have a less 
severe effect on extension variables than on a unique factor. 
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We instructed the LISREL program to produce a two-stage least square (TSLS) 
solution, instead of a maximum likelihood (ML) solution because the ML solution would have 
been unreasonably costly. Previous studies have indicated that the TSLS estimates are 
sufficiently close to the ML estimates, and that the additional cost of an ML analysis is 
generally not warranted. 

Table 10 gives the estimated true-score correlations between subtests. According to the 
test design, the subtest would be construct valid if all subtests within CS were highly correlated 
with one another, and those subtests within GK were also highly intercorrelated. PK should not 
be highly correlated with any of the others, and GK factors should not correlate highly with CS 
factors. Correlations within the dotted lines in the table should be higher than the correlations 
outside of the dotted lines. 

First, looking at Mathematics, we see that its correlation with Science was higher than 
its correlation with any other subtest. It did not correlate highly, however, with other factors 
regarded as GK or with PK or CS factors. 

The Social Studies factor correlated better with the Literature and Fine Arts factor and 
with the Science factor than it did with PK or the three areas of CS. It did not correlate highly 
with Mathematics. 

Literature and Fine Arts, on the other hand, correlated about the same with the CS 
factors as it did with the other GK factors and rather poorly with Mathematics; it also 
correlated highly with PK. 

Science, while it correlated moderately well with Mathematics, correlated no better with 
other GK factors than it did with CS factors. 

This analysis of the GK factors suggests that Mathematics does not belong in GK but is 
probably a totally different factor from any of the others measured. Literature and Fine Arts 
as well as Science had as high a verbal load as the CS factors do. Statistically, they are 
indistinguishable. There is some evidence that Social Studies is somewhat different from either 
PK or CS and therefore has some discriminant validity. 

PK is very highly correlated with the CS factors and with Literature and Fine Arts and 
Science. In fact, PK is more highly correlated with the Reading factor than Reading is with 
Writing. 

The two essay ratings loaded better on the multiple-choice Writing subtest than on the 
other subtests, though Rating 1 loaded higher on Reading than on Writing. All of the essay 
factor loadings were low; the largest was 0.16, and that was the one loading on Reading. 

The only measure of fit of the model produced by the TSLS solution was a coefficient 
of determination. This number can range from 0 to 1 and may be regarded as the fraction of 
variance explained by the model. For this model it was 0.263. We would expect the fit to be 
good because there are a large number of factors. One condition that would reduce the degree 
to which the model fit the data would be if certain items loaded more highly on a different 
factor than on the one specified, for example, if a Fine Arts item actually loaded better on 
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Reading than it did on the factor containing the other Fine Arts items. It is likely that this was 
true of some items, considering the high correlations between certain factors, such and Reading 
and PK. 

In summary, the pattern of correlations in the eight-factor model suggests that 
Mathematics is something different from all other constructs, and that Literature and Fine 
Arts, Science, and Professional Knowledge are all indistinguishable from the three CS factors. 
These three subtests, in other words, do not show discriminant validity. Social studies, however, 
does seem to be more similar to Literature and Fine Arts and to Science than it is to the CS 
or PK factors. 

Table 11 shows the factor loadings for the eight-factor solution. 



The Three-Factor Model 

The solution to the eight-factor model did not suggest that subtests grouped into 
Communication Skills, General Knowledge, and Professional Knowledge as per test design. To 
estimate more exactly just how well items may load onto these three a priori factors, we set up 
that model for LISREL to test explicitly. 

The coefficient of determination was 0.147. Table 12a shows the estimated true-score 
correlations among those three factors. The correlations were quite high, especially between 
CS and PK, suggesting that the factors do not have high discriminant validity. 

Table 13 presents the factor loadings for this model. The essay ratings were allowed to 
load on the CS factor alone because, according to test design, the essay is part of CS. The 
loadings were only .16 and .14 for these ratings. This suggests that the essay ratings are either 
measuring something very different from CS, or they have low reliability, or both. 

The three-factor model solution alone is not very informative. It fits less well than the 
eight-factor model, but larger numbers of factors nearly always fit better than smaller models. 
What is more informative is the comparison of this model with a competing one. 



The Two-Factor Model 

Results of the eight-factor model suggested that there may be only two factors: 
mathematics and everything else. A comparison of the two-factor model with the three-factor 
model reported above would provide a test of two competing models: one in accordance with 
the a priori test design and one consistent with empirical correlations. 

For the two-factor model, we named the non-mathematics items "Verbal". Table 12b 
shows the results of the LISREL test of the two-factor model. 

Verbal and Mathematics factors were correlated 0.78, a value that shows considerably 
more discriminant validity than the correlations between CS, GK, and PK in the three-factor 
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model. The coefficient of determination was 0.219, a figure that was actually higher for the 
two-factor model than for the three- factor model. All other things being equal, we would 
expect a two-factor model to provide a poorer fit to the data than a three-factor model. The 
fact that the two-factor model fit better is evidence in its favor that would be difficult to 
dispute. 

Table 14 gives the factor loadings for the two-factor model. The essay ratings loaded 
very little on either factor: 0.12 and 0.09 on Verbal; 0.10 and 0.04 on Mathematics. These 
values were consistent with the earlier models showing the essay ratings to be measuring 
something different from the rest of the Battery. 

An examination of the other factor loadings showed that some items had near-zero 
loadings on the "Verbal" factor. Those with loadings less than 0.2 were Science item number 
23, Reading item number 1, and Listening items 3 and 28. An examination of the tetrachoric 
correlations showed that Science item 23 and Reading item 1 were uncorrelated with any other 
item and therefore may be measuring something quite different from the rest of the Battery, or 
they may be ambiguous or incorrectly keyed. 

Similarly, some items loaded quite heavily on the "Verbal" factor, the highest being 
0.834 for Social Studies item 17. Three other items loaded higher than 0.6. These were two 
Social Studies items and one from Science. 

In general, there seemed to be no pattern of Verbal factor loadings that we could relate 
to specific subtests. Reading, for example, did not load any higher on that factor than did 
Social Studies. 



Analysis Designs Attempting to Control for the Effects of Reading Skill 

It was clear from the outcomes of the analyses conducted thus far that performance on 
the non-mathematical items in the Battery was highly related to a single factor, probably 
reading ability or general verbal skills. We therefore designed two types of analyses attempting 
to control for reading ability. The first was to divide the population into high-scoring and low- 
scoring readers with the intention of testing all models on those two groups separately. If 
reading ability limits performance on other subtests, those other subtests should be more 
construct valid for high readers than for low readers. 

What we found, and discussed earlier, was that the restriction in range of Reading 
scores among high-scoring readers prevented a tetrachoric correlation from being computable. 
The correlation was computable on the low readers, however, so that group's data were 
analyzed. 

The eight-factor LISREL model resulted in findings similar to those obtained for the 
population as a whole, except that the Reading factor, probably because of the restricted range 
on reading scores, correlated near zero with other factors. Table 15 shows the estimated 
true-score correlations for this model. 



25 

31 



The three-factor and two-factor models for low readers produced results almost identical 
with those for the total population, except that all of the correlations were lower. 

It is difficult to interpret the findings from these low-reader models because the 
Communication Skills factor essentially excludes Reading. In other words, while we may have 
successfully removed Reading effects from the other factors, we also removed them from 
Communication Skills, which is uninterpretable. 

The LISREL program was not able to produce a coefficient of determination for any of 
the models using data from low readers. This problem may be reflecting the artificiality of 
restricting the range on one of the variables included in the analysis. 

In response to this possibility, we tried leaving reading scores out of the analysis 
altogether and testing the resulting seven-factor model on low readers. The LISREL program 
found this model to be unsolvable (the phi matrix was not positive definite). 

Our attempts to "remove" reading from our analyses were unsuccessful, either to remove 
it empirically or statistically. We then developed a totally different strategy to understand 
whether some items more than others were responsible for the high verbal load present in each 
subtest. 



The Common Reading Factor Model 

The persistent finding that reading or other verbal skills seemed to affect performance 
on Professional Knowledge and the General Knowledge subtests with the exception of 
Mathematics led us to the development of a model in which each item (with the exception of 
mathematics) was hypothesized to measure reading ability plus some other ability identified by 
its subtest. In other words, the Science subtest should measure reading ability as well as 
knowledge of science. The model allowed each item on the "Verbal" factor to load on two 
factors, with the exception of Reading, which was set to load only on the Reading factor. In 
addition, we required that the Reading factor be uncorrelated with each of the other factors, 
though the remaining factors could be correlated with one another. 

This design seemed to reflect the real world most accurately. Performance on a Science 
item should be influenced by two things: reading ability and science knowledge. Similarly, 
performanence on a Professional Knowledge item should be influenced by reading ability and 
knowledge of the profession of education. We might expect some correlation, however, 
between knowledge of science and professional knowledge, or at least we would not deny its 
possibility. 

The solution to such a model would allow us to look at each item individually and 
determine the extent to which performance on that item is affected by reading ability and the 
extent to which it is affected by the knowledge it was designed to measure. Those items with 
higher loadings on the Reading factor than on the subtest factor would be designated poor 
items because they reduce the construct validity of the test. Those items with higher loadings 
on the specific subtest factor than on Reading would be working he way they were designed to 
work. A followup content analysis cc >ld possibly uncover the differences between these two 
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types of items. Knowing the differences, we could design the next version of the Battery to 
contain more construct valid items. 

After a great many attempts to "fine tune" the model, the LISREL program was unable 
to produce any solution. The model variations included removal of the essay ratings from the 
model, specification of the essay as a factor of its own, and instructions to produce an 
unweighted least squares solution instead of a two-stage least squares solution. When the 
LISREL program did produce initial estimates, the phi matrix was not positive definite and the 
estimates of phi were out of range (correlations much greater than 1.0). The estimated factor 
loadings on Reading, Mathematics, and Social Studies were reasonable, however, while on the 
other factors the loadings were near zero. 

What these results may suggest is that there was virtually no reliable variance, beyond 
reading ability, in the scores of items designed to measure other skills. Consequently, a model 
designed to estimate the amount of that unique reliable variance could not have a solution. 



Conclusions of Part II 

The exploratory factor analysis (using TESTFACT) as well as the confirmatory factor 
analyses (using LISREL) pointed to the conclusion that the multiple-choice items in the Core 
Battery were measuring a large general factor, probably verbal or reading skill, plus a 
mathematics factor. Social Studies showed some unique variance, but since General Knowledge 
was not defined by the other factors (they were indistinguishable from Communication Skills), 
it is difficult to argue that Social Studies was part of General Knowledge. The essay ratings 
were measuring something quite different from either the verbal or the mathematics factors, 
though they were more highly correlated with verbal than mathematics. 

Attempts to extract reading ability from the items designed to measure other skills, 
namely, Science, Social Studies, Literature/Fine Arts, and Professional Knowledge, resulted in 
models that were unsolvable, apparently because there was too little unique variance in the 
scores on those items. 

In summary, the eight-factor model showed the intercorrelations of factors within a 
content area (CS or GK) to be generally no higher than correlations between content areas that 
were designed to measure different constructs. The solution to that model implies that the 
Battery is not construct valid if the constructs are defined to be CS, GK, and PK. The test of 
the two-factor (V and M) model versus the three-factor (GK, PK, and CS) model showed the 
two-factor model to fit better than the three-factor model, again supporting a different pattern 
of constructs than was intended by the test design. 
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DISCUSSION 



Lack of discriminant validity has important practical implications. When we report test 
performance to an examinee, regardless of whether it is a scaled score or number right or 
whatever, implicit in that report is a claim that the score represents the level of the examinee's 
knowledge in that subject area. If we call it Professional Knowledge, then we are implying that 
it is not Reading or Science. More importantly, it is misleading to the examinee to be given 
one score in what claims to be Professional Knowledge and another score in what claims to be 
Science if the difference between those scores can be attributed entirely (or almost entirely) to 
random measurement error. This is unfortunately what is occurring with the seven subtests we 
have called "verbal skills." 

There are a number of directions that the NTE Program might take to address their 
problems with construct validity, and we will discuss some of them here. They are all fairly 
straigtforward technically but will require some commitment to making changes, not only in the 
method of score reporting, but in the test itself. Because the test will be revised in the near 
future, none of the suggestions outlined here would incur great expense or cause a complete 
upheaval of the Program. 

The first suggestion is quite simple, namely, that the Mathematics subtest be removed 
conceptually from General Knowledge. It makes no sense from the viewpoint of education or 
cognitive psychology to group mathematics with literature and then to expect the two of them 
to be different from reading. In no other testing program is mathematics regarded as a 
knowledge area along side of fine arts, literature, science, and social studies-all forming one 
factor (higher order or otherwise). By calling the Mathematics subtest "mathematical ability," 
or better yet, "quantitative skills," and reporting quantitative scores separately from all other 
scores, one major problem in construct validity is solved without making any changes to the test 
or conducting any studies. 

The second suggestion is directed toward understanding why the items on seven 
subtests, supposedly measuring different constructs, all seem to measure only one. This will 
involve a statistical analysis followed by a content analysis. In the standard development of a 
test, an "item analysis" is performed to identify those items that do not correlate well with the 
total test score or subscore. Biserial correlations are computed between each item score and 
the total scale score. 

A modification of that procedure in which each item on all seven "verbal" subtests is 
correlated with the total Reading score would identify items that are primarily reading items. 
The remaining items could then be grouped and given subtest scores which should correlate less 
highly with Reading. This procedure would be especially profitable for Professional Knowledge 
which has such a high correlation with Reading. If there is still a high correlation between 
other subtests, such as Science and Social Studies, then results of these tests should be reported 
together, as General Knowledge, without trying to break them down into more specific subjects. 

After identifying the items correlating highly with reading, content experts could 
examine the "high reading" items and compare them with the items correlating less well with 
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reading to see what the differences are. This information could be applied in future revisions 
of the Core Battery. 

Mislevy has suggested a third approach to dealing with lack of discriminant validity by 
modifying the reporting system so as to provide candidates with empirical Bayes confidence 
bands that they can use to make decisions about retesting. This information would tell them 
whether their subscores are significantly different (thus enabling them to understand their 
relative strengths and weaknesses), or whether the observed differences in their subscores are 
simply random fluctuations. This information could affect candidate decisions regarding retest 
preparation. 

One final suggestion is that the Core Battery be tested on samples of students or 
professionals who have never taken an education course in order to see if some evidence of 
construct validity can be obtained for the Professional Knowledge subtest. This technique is 
often used to provide construct validity for occupational tests. An appropriate design would be 
to select three samples, perhaps seniors in engineering, law students, and practicing writers or 
journalists. Each group would be strong in some area that is supposedly measured by the Core 
Battery, and none would be expected to score well on Professional Knowledge. If the profiles 
of these three groups (or any number of groups) could be distinguished by their scores on the 
Core Battery, there would be good evidence both of convergent and discriminant validity. 

We have focused most of our attention in this report on discriminant validity because it 
could be analyzed very thoroughly. Evidence for convergent validity is more difficult because, 
in Part I, we had only one variable other than test scores with which to link those scores, and in 
Part II we had no external variables. 

We found in the analyses in Part I that undergraduate GPA correlated moderately with 
each of the three test factors. If we had had grades in specific subjects corresponding with the 
test content areas, we might have been able to establish more definite convergent validity. We 
do not know, for example, what our large verbal factor is really measuring. It may be reading, 
reasoning, test-taking skill, or some combination of these. We do not know what the essay is 
measuring -- only that it is not measuring what the other subtests measure. If we had a grade 
in English Composition to correlate with it, for example, we might find some evidence for what 
it does measure. 

Likewise, we assume that the Mathematics subtest measures mathematical skills, but we 
have no evidence in the form of other mathematics tests or grades in mathematics to support 
that claim. Therefore, we have not established convergent validity because we did not have the 
measures we needed to do so. The only evidence of convergent validity we did find was that 
performance on all of the subtests is positively correlated with undergraduate grades. We can 
conclude, therefore, that the Core Battery as a whole measures some of the same things 
reflected in students' grades, but we do not know whether it is connected with specific course 
content or whether it is an indicator of general intelligence and academic performance. 

A fifth recommendation, therefore, is to obtain external measures of each construct so 
that a stronger and more complete construct validity model can be tested. 
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One final comment addresses a contradiction inherent in the intent of the Core Battery, 
a contradiction that has arisen because of the mixed criticisms that have been directed agains 
the test since its beginning and the attempts on the part of the NTE Program to satisfy all of 
the critics. On the one hand, many "experts" want a test that measures general academic skills 
without requiring a knowledge of subject content. After all, they claim, that is why the NTE 
has Area tests. A second faction of "experts" wants the Core Battery to be structured according 
to subject content, with detailed diagnostic feedback to the candidates. But to satisfy the first 
group of experts, the items were designed so that subject knowledge was not required in order 
to answer the items. To satisfy the second group, diagnostic information is being reported to 
the candidate. But to provide diagnostic information requires that the subtests be measuring 
different content and that construct validity studies be done to demonstrate that they measure 
different content. 

These two very different views on the part of "experts" place contradictory demands on 
the NTE Program. Perhaps the Program should take its own stand, based on resident expertise 
in teaching and in psychometrics, and construct a test based on that commitment. The NTE 
Program does not have to serve many masters. Neither the exam nor its purpose needs to be 
complicated. It simply has to be a competently constructed, reasonably validated examination, 
designed to accomplish a clear and consistent mission. 
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Table 1 

Means and Standard Deviations of Subtest Raw Scores and GPA 



Means 



Maximum 
Possible 


White 
Males 


White 
Females 


Black 
Males 


Black 
Females 


Subtest Score 


(n=791) 


(n=3,797) 


(n=97) 


(n=498) 


Listening 40 


29.71 


30.39 


22.42 


22.16 


Reading 30 


20.24 


20.67 


13.24 


13.12 


Writing (multi. choice) 45 


27.18 


29.33 


18.72 


20.55 


Essay 12 


6.98 


7.76 


5.04 


5.94 


Lit. /Arts 35 


24.15 


24.65 


15.94 


16.59 


Mathematics 25 


17.25 


15.68 


10.55 


8.93 


Science 30 


20.75 


19.50 


12.87 


12.19 


Social Studies 30 


20.68 


19.34 


13.63 


13.11 


Prof. Knowledge 105 
(sum of 3 parts) 


64.46 


69.46 


44.42 


48.42 


GPA to Date* 3.75 


2.94 


3.15 
Standard 


2.82 
Deviations 


2.93 




White 
Males 


White 
Females 


Black 
Males 


Black 
Females 


Listening 


5.74 


5.15 


5.40 


5.95 


Reading 


5.30 


4.71 


4.57 


4.89 


Writing (multi. choice) 


7.38 


6.48 


5.65 


6.03 


Essay 


1.88 


1.64 


1.82 


1.94 


Lit. /Arts 


5.88 


5.21 


5.11 


5.43 


Mathematics 


4.48 


4.39 


4.31 


3.77 


Science 


5.00 


4.44 


4.20 


4.14 


Social Studies 


5.16 


4.65 


4.42 


4.39 


Pro! . Knowledge 
(sum of 3 parts) 


15.39 


12.64 


12.45 


13.39 


GPA to Date* 


36 °- 45 


0.45 


0.40 


0.43 



JC'GPA is estimated by using the mid-point of the intervals designated as response alternativi 



Table 2 



Subtest 



Estimated True-Score Correlations* 
(N = 5,392) 



(1) (2) (3) (4) (5) (6) (7) (8) (9) 



(1) Listening 1.000 0.94O 0.841 0.869 0.847 0.865 0.860 - 0.801 

(2) Reading 0.924 1.000 0.879 0.916 0.898 0.911 0.919 - 0.828 

(3) Writing (multi. choice) 0.822 0.867 1.000 0.883 0.820 0.827 0.843 - 0.763 

(4) Lit. /Art 0.851 0.899 0.866 1.000 0.902 0.921 0.882 - 0.774 

(5) Science 0.835 0.892 0.812 0.891 1.000 0.937 0.874 - 0.883 

(6) Social Studies 0.843 0.895 0.810 0.902 0.926 1.000 0.883 - 0.825 

(7) Prof. Knowledge 0.848 0.912 0.834 0.873 0.875 0.871 1.000 - 0.748 
(all parts) 

(8) Essay** 0.592 0.628 0.674 0.627 0.546 0.551 0.614 1.000 

(9 ) Ma th 0.779 0.812 0. -7 45 0.755 0.871 0.806 0.734 0.490 1.000 



♦Correlations above the diagonal are based on reported estimates from the Test Analysis of the 
NTE Core Battery, Form 3ENT, SR-84-19. Correlations below the diagonal are based on the 
nine-factor model for all subgroups combined. 

** 

Estimate is based on corrections for attenuation using interrater reliabilities, not score 
reliabilities which would require at least two essays to estimate. True correlations with 
the essay factor may be higher. 
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Table 3 







Reliability Estimates 


from the 


Three-Factor 


Model 












Reliability 


— 


Factor 




Siab test 


White 
Males 


White 
Females 


Black 
Males 


Black 
Females 






Listening 


0.712 


0.686 


0.662 


0.715 






Reading 


0.790 


0.744 


0.665 


0.749 






Writing (multi . choice) 


0.730 


0.713 


0.588 


0.681 


VERBAL 


< 


Lit. /Art 


0.784 


0.731 


0.686 


0.706 






Science 


0.735 


0.668 


0.604 


0.569 






Social Studies 


0.774 


0.705 


0.706 


0.672 






Prof. Knowledge 
(all parts) 


0.867 


0.831 


0.808 


0.842 


ESSAY * 




Essay 


0.732 


0.660 


0.775 


0.744 


MATH 




Math 


0.814 


0.791 


0.804 


0.646 



k Reliability estimates are based solely on interrater reliabilities, not score 
reliabilities which would require at least two essays. 



ERJC 



Table 4 



Reliability Estimates Based on Reported Values of KR-20* 



Subtest Reliability 

Listening 0.840 

Reading 0 . 840 

Writing (multi. choice) 0.857 

Lit. /Art 0.840 

Science 0.798 

Social Studies 0.801 

Prof. Knowledge 0.919 

Essay 

Math 0.831 



*Reported in the Test Analysis of the NTE Core Battery, Form 3ENT, February, 1984, SR-84-19. 
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Table 5 



Reliability Estimates Based upon the Nine-Factor Model or 

upon Split Halves 



Reliability 



Subtest 
Listening 
Reading 

Writing (multi. choice) 

Lit. /Art 

Science 

Social Studies 

Prof. Knowledge 
(all parts) 

Essay* 



White 
Males 

0.821 

0.819 

0.844 

0.847 

0.801 
0.831 

0.911 
fc.732 



White 
Females 

0.791 
0.787 
0.834 
0.800 
0.731 
0.769 
0.880 

0.660 



Black 
Males 

0.732 

0.719 

0.773 

0.788 

0.652 

0.746 

0.855 

0.784 



Black 
Females 

0.808 
0.754 
0.781 
0.768 
0.646 
0.725 
0.879 

0.744 



Math 



0.813 



0.790 



0.807 



0.646 



* 

Based solely on interrator reliability. 
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Table 6a 



Components of Variance of Scores on Selected Subtests 

White Males Only 



Percent of total variance explained by 



Subtest 
Listening 
Reading 

Writing (multi. choice) 

Lit. /Art 

Science 

Social Studies 

Prof . Knowledge 
(sum of 3 parts) 



general 
verbal 
ability 

71.2 

79.0 

73.0 

78.4 

73.5 

77.4 

86.7 



random 
measurement 
error 

17.9 

18.1 

15.6 

15.3 

19.9 

16.9 

8.9 



uniqueness 
of 
subtest 

10.9 

2.9 
11.4 

6.3 

6.6 

5.7 

4.4 
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Table 6b 



Components of Variance of Scores on Selected Subtests 
White Females Only 



Percent of total variance explained by 





general 


random 


uniqueness 




verbal 


measurement 


of 


Subtest 


ability 


error 


subtest 


Listening 


68.6 


20.9 


10.5 


Reading 


74.4 


21.3 


4.3 


Writing (multi. choice) 


71.3 


16.6 


12.1 


Lit. /Art 


73.1 


19.9 


7.0 


Science 


66.8 


26.9 


6.4 


Social Studies 


70.5 


23.1 


6.4 


Prof . Knowledge 


83.1 


12.0 


4.9 



(sum of 3 parts) 
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Table 6c 



Components of Variance of Scores on Selected Subtests 

Black Males Only 



Percent of total variance explained by 



Subtest 
Listening 
Reading 

Writing (multi. choice) 

Lit. /Art 

Science 

Social Studies 

Pro f . Knowl edge 
(sum of 3 parts) 



general 
verbal 
ability 


random 
measurement 
error 


uniqueness 
of 
subtest 


66.2 


26.8 


7.0 


66.5 


28.1 


5.4 


58.8 


22.7 


18.5 


68.6 


21.2 


10.2 


60.4 


34.8 


4.8 


70.6 


25.4 


4.0 


80.8 


14.5 


4.7 
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Table 6d 



Components of Variance of Scores on Selected Subtests 
Black Females Only 



Percent of total variance explained by 



Subtest 


general 
verbal 
ability 


random 
measurement 
error 


uniqueness 
of 
subtest 


Listening 


71.5 


19.2 


9.3 


Reading 


74.9 


24.6 


0.5 


Writing (multi. choice) 


68.1 


21.7 


10.2 


Lit ./Art 


70.6 


23.2 


6.2 


Science 


56.9 


35.4 


7.7 


Social Studies 


67.2 


27.5 


5.3 


Prof . Knowledge 


84.2 


12.1 


3.7 



(sum of 3 parts) 
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Table 7 



E sti M ted True-Soore Correlations between Three B> Core Battery rectors and CPA* 



Group 



White 
Males 



Verbal 
Essay 
Math 
GPA 



Verbal 

1.000 
0.660 
0.790 
0.408 



Essay 

1.000 
0.479 
0.300 



Math 



1.000 
0.265 



GPA 



1.000 



White 
Females 



Verbal 
Essay 
Math 
GPA 



1.000 
0.438 
0.707 
0.280 



1.000 
0.361 
0.217 



1.000 
0.213 



1.000 



Black 
Males 



Verbal 
Essay 
Math 
GPA 



1.000 
0.529 
0.795 
0.439 



1.000 
0.401 
0.236 



1.000 
0.348 



1.000 



Black 
Females 



Verbal 
Essay 
Math 
GPA 



1.000 
0.667 
0.770 
0.285 



1.000 
0.512 
0.222 



1.000 
0.183 



1.000 



Corrections for attenuation for the essay are based solely upon interrater 



reliabilities, not score reliabilities, 
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Table 8 



Standard Errors of Measurement Based on Three-Factor Model 



Subtest 
Listening 
Reading 

Writing (multi. choice) 

Lit. /Art 

Science 

Social Studies 

Prof . Knowledge 
(sum of 3 parts) 

Essay 



White 
Males 

3.09 

2.43 

3.83 

2.75 

2.58 

2.44 

5.61 

0.96 



White 
Females 

2.89 

2.39 

3.47 

2.73 

2.58 

2.53 

5.18 

0.96 



Black 
Males 

3.14 

2.64 

3.64 

2.86 

2.65 

2.42 

5.48 

0.84 



Black 
Females 

3.18 

2.45 

3.41 

2.77 

2.72 

2.52 

5.32 

0.98 



Math 



1.94 



2.01 



1.88 



2.24 



ERIC 
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Table 9 



PROMAX PRIMARY FACTOR LOADINGS 







1 




2 


3 






1 




2 


3 


GKSS 


1 


0. 


2737 


-0.1811 


0.0927 


PK#2 


1 


0. 


5100 


0.1241 


0.0155 


GKSS 


3 


0. 


3366 


-0.0902 


0.0901 


PK#2 


3 


0. 


3884 


0.1714 


0.0516 


GKSS 


5 


0. 


6693 


-0.0674 


0.1642 


PK#2 


6 


0. 


5116 


-0.1067 


0.0397 


GKSS 


7 


0. 


1262 


-0.1372 


-0.1582 


PK#2 


8 


0. 


3558 


0.0621 


-0.0396 


GKSS 


9 


0. 


2719 


-0.3833 


0.1683 


PK#2 


11 


0. 


3620 


' 0.0629 


0.1021 


GKSS 


11 


0. 


3120 


-0.2302 


0.0267 


PK#2 


13 


0. 


4099 


0.0615 


0.0121 


GKSS 


13 


0. 


3157 


-0.3044 


0.1293 


PK#2 


16 


0.4917 


0.0467 


0.0340 


GKSS 


15 


0.4955 


-0.1038 


0.0792 


PK#2 


18 


0. 


6409 


0.0256 


0.0846 


GKSS 


17 


0. 


4517 


-0.5548 


0.1212 


PK#2 


21 


0. 


2658 


0.1056 


0.0629 


GKSS 


19 


0. 


3764 


-0.1886 


0.1364 


PK#2 


23 


0. 


1787 


-0.0945 


-0.0316 


GKSS 


21 


0. 


4199 


-0.3566 


0.1021 


PK#2 


26 


0. 


4162 


-0.0322 


.0151 


GKSS 


24 


0. 


4592 


-0.2635 


-0.0036 


PK#2 


28 


0. 


4036 


-0.0166 


-0.0073 


GKSS 


25 


0. 


3724 


-0.1531 


-0.0160 


PK#2 


31 


0. 


4060 


0.0067 


-0.0497 


GKSS 


27 


0. 


2622 


-0.3134 


-0.0103 


PK#2 


33 


0. 


3240 


0.1513 


0.0065 


GKSS 


29 


0. 


4100 


0.0870 


0.0154 


CSL 


1 


0. 


3198 


0.0742 


-0.0854 


GKM 


1 


0. 


2763 


-0.1464 


-0.3096 


CSL 


3 


0. 


1896 


-0.0269 


0.0035 


GKM 


3 


0. 


5658 


0.0718 


-0.2953 


CSL 


6 


0. 


5508 


0.2117 


0.0014 


GKM 


5 


0. 


1406 


-0.1638 


-0.2595 


CSL 


8 


0. 


4368 


0.1685 


-0.0356 


GKM 


6 


0. 


3529 


-0.0094 


-0.?074 


CSL 


11 


0. 


5515 


-0.0063 


0.0415 


GKM 


8 


0. 


0457 


-0.1764 


-0.1969 


CSL 


13 


0. 


5090 


-0.0174 


-0.0141 


GKM 


10 


0. 


4310 


-0.0451 


-0.3803 


CSL 


16 


0.4210 


0.0320 


-0.0538 


GKM 


11 


0 


3161 


0.0504 


-0.2543 


CSL 


18 


0. 


3407 


0.1058 


0.0406 


GKM 


13 


0 


3440 


-0.2606 


-0.3775 


CSL 


21 


0 


2625 


-0.0576 


0.1064 


GKM 


15 


0 


2849 


-0.1782 


-0.3442 


CSL 


23 


0.4510 


0.0821 


0.0042 


GKM 


16 


0 


3760 


0.0382 


-0.2230 


CSL 


26 


0 


5641 


0.2182 


-0.0090 


GKM 


18 


0 


3759 


-0.0880 


-0.3048 


CSL 


28 


0 


0975 


-0.0130 


-0.0220 


GKM 


20 


0 


2700 


-0.0829 


-0.2992 


CSL 


31 


0 


5418 


0.2227 


-0.0613 


GKM 


21 


0 


3493 


-0.1347 


-0.3950 


CSL 


33 


0 


5494 


0.1116 


0.1251 


GKM 


23 


0 


3578 


-0.2157 


-0.3718 


CSL 


36 


0.4315 


-0.0568 


0.0268 


GKM 


25 


0 


2561 


0.0733 


-0.2796 


CSL 


38 


0 


4635 


-0.0209 


0.0360 


GKLA 


1 


0 


3302 


-0.0346 


-0.0018 


CSR 


1 


0 


0835 


0.0323 


-0.0016 


GKIA 


3 


0 


3929 


-0.0253 


0.0656 


CSR 


3 


0 


.5983 


0.0767 


0.0376 


GKLA 


7 


0 


4247 


0.0424 


0.0245 


CSR 


5 


0.4387 


-0.0729 


-0.0270 


GKLA 


8 


0 


2719 


-0.1886 


-0.0295 


CSR 


7 


0 


2231 


-0.0056 


0.0037 


GKLA 


11 


0 


3134 


-0.1249 


0.0553 


CSR 


9 


0 


.5657 


0.0582 


-0.0063 


GKLA 


13 


0 


5006 


-0.0632 


0.0648 


CSR 


11 


0 


.6454 


0.0240 


0.0598 


GKLA 


16 


0 


.4319 


-0.0274 


0.0319 


CSR 


13 


0.4843 


0.0134 


0.0940 


GKLA 


18 


0 


.4717 


-0.1215 


0.0684 


CSR 


15 


0 


.2790 


-0.1113 


-0.0587 


GKLA 


21 


0 


.3201 


-0.0062 


-0.0078 


CSR 


17 


0 


.3760 


0.0195 


-0.0064 


GKLA 


23 


0 


.1519 


-0.0703 


-0.0265 


CSR 


19 


0 


.5316 


0.1208 


-0.0150 


GKLA 


26 


0 


.3618 


-0.0373 


0.0017 


CSR 


21 


0 


.7115 


0.1603 


0.0301 


GKLA 


28 


0 


.4200 


0.0394 


0.0622 


CSR 


23 


0 


.5263 


0.1944 


0.0328 


GKLA 


31 


0 


.2653 


-0.0439 


0.0581 


CSR 


25 


0 


.5287 


0.1632 


-0.0580 


GKLA 


33 


0 


.3151 


0.0693 


-0.0669 


CSR 


27 


0 


.5284 


0.0659 


-0.1295 


GKS 


1 


0 


.3296 


-0.0168 


0.0149 


CSR 


29 


0 


.3981 


0.1697 


-0.0210 


GKS 


3 


0 


.2534 


-0.0602 


-0.0543 


CSW 


1 


0 


.5482 


0.1531 


-0.0359 


GKS 


5 


0.5243 


-0.1863 


-0.0686 


CSW 


4 


0 


.3346 


0.0555 


-0.0770 


GKS 


7 


0 


.1312 


-0.1440 


-0.1019 


CSW 


7 


0 


.2287 


-0.1152 


-0.0010 


GKS 


9 


0 


.6412 


-0.1183 


0.0302 


CSV 


10 


0 


.2333 


-0.0912 


-0.0329 


GKS 


11 


0 


.3721 


-0.2189 


-0.0473 


CSW 


13 


0 


.3093 


-0.0525 


-0.0402 


GKS 


13 


0 


.3132 


0.0413 


-0.0022 


CSW 


16 


0 


.3164 


0.0915 


-0.0264 


GKS 


15 


0 


.3526 


-0.2519 


-0.0085 


CSW 


19 


0 


.2540 


-0.0385 


-0.0537 


GKS 


17 


0 


.3358 


-0.1476 


-0.0765 


CSW 


22 


0 


.2789 


0.0391 


-0.1326 


GKS 


19 


0 


.3677 


-0.2262 


0.0012 


CSW 


25 


0 


.2653 


-0.0926 


-0.0323 


GKS 


21 


0.1161 


-0.1499 


-0.1670 


CSW 


28 


0 


.5500 


0.0781 


-0.0154 


GKS 


23 


0 


.0195 


0.1066 


0.0041 


CSW 


31 


0 


.4286 


0.0007 


0.0149 


GKS 


25 


0 


.5046 


-0.0737 


0.0101 


CSW 


34 


0 


.5973 


0.1734 


-0.0192 


GKS 


27 


0 


.2870 


-0.1101 


-0.0286 


CSW 


37 


0.4087 


-0.0531 


0.0186 


GKS 


29 


0 


.3854 


-0.0389 


-0.0992 


CSW 


40 


0 


.4455 


0.0994 


-0.0440 














CSW 


43 


0.4384 


0.1250 


-0.0274 



o 
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Table 10 

Estimated True-Score Correlations 
(N = 13,059) 







(1) 


(2) 


(3) 


(4) 


(5) 


(1) 


Social Studies 


Jl.000 " 










(2) 


Mathematics 


| .655 


1.000^ 


















, General 


(3) 


Lit/Fine Arts 


\ .850 


.717 


1.000" 


■«K. 




(4) 


Science 


i .879 


.799 


.869 


1.000 v 




(5) 


Prof Knowledge 


.748 


.715 


.864 


.860 


1.000 


(6) 


Listening 


.714 


.717 


.879 


.817 


.899 


(7) 


Reading 


.733 


.738 


.881 


.830 


.920 


(3) 


Writing (m/c) 


.701 


.735 


.883 


.786 


.858 



(6) (7) (8) 



1.000 v ^ 
.939 1.000 
.866 .888 



Communication 
Skills 



1.000 



Dotted lines enclose correlations that should be high compared with 
correlations outside of the dotted lines in order for the test to be valid as a 
measure of three constructs: Communication Skills, General Knowledge, and 
Professional Knowledge. High correlations inside the lines would indicate 
convergent validity; low correlations outside the lines would indicate 
discriminant validity. 



Table 11 

Standardized Factor Loadings for Eight-Factor Model" 

(N = 13,059) 





SCC- STUD 


— 


L I T — A ft T 


SC \ ENCE 


PROF— KN 




REtQISG 
0.0 


URITE-HC 


V A K 1 


U •HOC 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 2 


0*332 


0.0 


0.0 


CO 


O.C 


0.0 


0.0 


0.0 


V A R 3 


0 • 549 


0.0 


CO 


c. c 


0.0 


0.0 


0.0 


0.0 


VAR 4 


0 • 206 


0.0 


0.0 


0.0 


c.c 


0.0 


0.0 


0.0 


VAR 5 


0*431 


0.0 


0.0 


0.0 


o.c 


0.0 


0.0 


0.0 


V A R 6 


0 m 576 


CO 


0.0 


0.0 


0. 0 


0.0 


0.0 


0.0 


uiB 7 

Y AK r 


U • *# f J 


A n 


0. 0 


CO 


CO 


0.0 


CO 


0.0 


VAR 8 


u • *» JH 


n ft 


n n 


u . u 


0.0 


0. 0 


0.0 


0.0 


VAR 9 


n t < 5 


CO 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 10 


n l *\ p. 


n ft 


A A 


0 . U 


0.0 


0. 0 


0.0 


0.0 


v A K II 


iT (An 


n ft 


n A 
Urn U 


0.0 


0. 0 


0. 0 


0.0 


0.0 


WAS 13 
YAK I £ 


U • 3 3 c 


ft ft 


n A 


CO 


0.0 


0.0 


0.0 


0.0 


VAR 1 3 


n T 07 
U • J w f 


n ft 


f: A 


ft 


0. 0 


0. 0 


0.0 


0.0 


U ft St 1 A 


n z. ai 


n ft 


n A 
Urn U 


0.0 


0.0 


0. 0 


0.0 


0.0 


YAK 13 




0.0 


0. 0 


CO 


0. 0 


0.0 


0.0 


0.0 


VAR 1 6 


n n 
u • u 


ft Afl3 


n A 


U.Li 


0.0 


0. 0 


0.0 


0.0 


WAS 17 

yak if 


n ft 
u • u 


0 . 609 


0*0 


0.0 


0.0 


0.0 


0.0 


0.0 


V AR 1 8 


0*0 


0,372 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


WAS 1 O 


n n 
u • u 


0*499 


A n 


CO 


0.0 


0. 0 


0.0 


0.0 


was 3fi 

YAK CU 


n i*i 
U »u 


n ">l & 
U . £**o 


A A 


0.0 


0.0 


0. 0 


0.0 


0.0 


U A O 31 
V AK CI 


n n 
U • U 


n a i "* 
UiO j j 


0.0 


CO 


0.0 


0.0 


0.0 


0.0 


U ft D 7>3 
V AK 


0*0 


0.392 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 23 


0 • C 


0.691 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 24 


CO 


0. 546 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 25 


0*0 


0 • 397 


0.0 


0.0 


0.0 


0. 0 


0.0 


0.0 


W ft D 3 A 
Y AK CO 


0*0 


ft CTl 
U • 3 he 


0.0 


0.0 


0.0 


0. 0 


0.0 


0.0 


W ft D 7 7 
YAK Cf 


A n 


0o452 


0. 0 


0.0 


0.0 


0.0 


0.0 


0.0 


u ft o 3 ft 

YAK CO 


0*0 


ft CttC 
U • 30 3 


0.0 


0.0 


0.0 


0. 0 


0.0 


0.0 


V AR 29 


0*0 


A car 
0* 373 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 30 


n a 


0 . 333 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


WAD 11 
Y*t K 31 


U • U 


ft A 


0.5 29 


0.0 


0.0 


0.0 


0.0 


0.0 


WAD "f. 3 
YAK JC 


a n 


n A 


0.371 


0.0 


0.0 


0. 0 


0.0 


0.0 


W ft 3 tl 
YAK JJ 


a n 


0.0 


0. 399 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 34 


0 • 0 


0.0 


0.319 


0.0 


0.0 


0. 0 


CO 


0.0 


VAR 35 


0*0 


0.0 


0.347 


0.0 


0.0 


0. 0 


0.0 


0.0 


VAR 36 


a n 


0.0 


0.494 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 37 


o.o 


0.0 


0.406 


n .1 
u . u 


u . u 


0. 0 


U.J 


0.0 


VAR 38 


0.0 


0.0 


0.47d 


0.0 


0.3 


0.0 


o.o 


0.0 


VAR 39 


0.0 


0.0 


0.127 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 40 


0.0 


0.0 


0.1 JO 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 41 


0.0 


0.0 


0.333 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 42 


0.0 


0.0 


0.437 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 43 


0.0 


0.0 


0.274 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 44 


0.0 


0.0 


0.359 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 45 


0,0 


3.0 


0.0 


0.474 


0.0 


0.0 


0.0 


0.0 


VAR 46 


f .c 


0.0 


0.0 


0.303 


0.0 


0. 0 


0.0 


0.0 


VAR 47 


0.0 


0.0 


0.0 


0.569 


0.0 


0.0 


0.0 


0.0 


VAR 48 


0.0 


o.c 


0.0 


0.173 


CO 


0.0 


0.0 


0.0 


VAR 49 


0.0 


0.0 


0.0 


0.679 


0.0 


0.0 


0.0 


0.0 


VAR 50 


0.0 


o.c 


0.0 


0.447 


0.0 


0.0 


0.0 


0.0 


VAR 51 


J.O 


CO 


0.0 


0.334 


0.0 


0.0 


0.3 


0.0 


V*R 52 


0.0 


0.0 


0.0 


0.405 


0.0 


0.0 


0.0 


0.0 


VAR 53 


0.0 


0.0 


0.0 


0.433 


0.0 


0.0 


0.0 


0.0 


VAR 54 


o.c 


0.3 


0.0 


0.430 


o.c 


0.0 


0.0 


0.0 


VAR 55 


0.0 


0.0 


0.0 


0.207 


0.0 


0.0 


■ 0.0 


■ 0.0 


VAR So 


0.0 


0.0 


0.0 


-0.008 


0.0 


0.0 


0.0 


0.0 


VAR 57 


0.0 


3.3 


0.0 


0.540 


0.0 


0.0 


0.0 


0.0 


VAR 53 


0.J 


0.0 


C.J 


0. 335 


0.0 


0.0 


0.0 


0.0 


VAR 59 


O.G 


o.o 


0.0 


0.421 


0.0 


0.0 


0.0 


0.0 


VAR 60 


o.c 


0.0 


0.0 


0.0 


0.642 


0.0 


0.0 


0.0 
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See Table 9 for translation of variable numbers to Core Battery items. 



Table 11 (Continued) 





SOCSTUJ 
J.- 


MATH 


LI 1 A K 1 




P RQ F~ ft W 




«Aii o1 


o.: 


J • J 


CO 


C J9o 


0.0 


VAR 62 


0. 3 


0.0 


0.0 


0.3 


0.501 


0.0 


VAR 63 


0.0 


0.0 


0.3 


0.0 


0.352 


0.0 


VAR 64 


0.0 


0.0 


0.0 


0.0 


0. 361 


0. 3 


VAR 65 


0.0 


0.0 


0.0 


0.0 


0.40S 


0.0 


VAR 66 


O.C 


0.0 


0.0 


0.0 


0.496 


0.0 


VAR 67 


0.0 


0.0 


0. 3 


0.0 


G.o20 


0.0 


VAR 08 


0.0 


o.c 


CO 


CO 


0.287 


0.0 


VAR 69 


0.0 


3.0 


0.0 


0.0 


0.182 


0.3 


VAR 70 


0.0 


0.0 


0.0 


0.0 


0.405 


0.0 


VAR 71 


0.0 


3.0 


0.0 


O.G 


0.396 


0.0 


VAR 72 


C.3 


o.c 


0.0 


0.0 


0.41 2 


0.0 


VAR 73 


0.0 


0.0 


0.0 


0.0 


0.340 


0.3 


VAR 74 


0.0 


0.0 


0.0 


0.0 


o.c 


0.551 


VAR 75 


CO 


3.0 


0.0 


0.0 


0.0 


0.1 64 


VAR 76 


o.c 


0.0 


0.0 


0.0 


0.0 


0.565 


VAR 77 


0.0 


0.0 


0.0 


0.0 


CO 


0. 449 


VAR 76 


0.0 


0.0 


0.0 


0.0 


0.0 


0.S44 


VAR 79 


0.0 


0.0 


0.0 


0.0 


0.0 


0.51 3 


VAR so 


0.0 


0.0 


0.0 


CO 


0.0 


0.41 0 


VAR 31 


0.0 


0.0 


0.0 


0.0 


0.0 


0.281 


VAR 82 


0.0 


0.0 


0.0 


0.0 


0.0 


0.2 34 


VAR 83 


0.0 


0.0 


0.0 


0.0 


0.0 


0.440 


VAR 84 


0.0 


3.0 


0.0 


0.0 


o.c 


0.583 


VAR dS 


0.0 


0.0 


0.0 


CO 


0.0 


0.117 


VAR 86 


O.G 


0.0 


0.0 


CO 


0.0 


0.580 


VAR 87 


0.0 


0.0 


0.3 


0.0 


o.c 


0.489 


VAR 88 


0.0 


3.0 


CO 


0.0 


0.0 


0.408 


VAR 89 


0.0 


o.c 


3.0 


0.0 


o.c 


C.442 


VAR 90 


0.0 


0.0 


0.0 


0.0 


0.0 


C. 0 


VAR 91 


0.0 


0.0 


CO 


0.0 


0.0 


0.3 


VAR 92 


0.0 


0.0 


0.0 


0.0 


o.c 


0.0 


VAR 93 


0.0 


0.0 


0.0 


0.0 


o.c 


0.0 


VAR 94 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 95 


0.0 


0.0 


0.3 


CO 


0.0 


0.0 


VAR 96 


o.c 


0.0 


0.0 


0.0 


o.c 


o.c 


VAR 97 


0.0 


0.0 


CO 


0.0 


CO 


0.0 


VAR 98 


0.0 


0.0 


0.0 


0.0 


o.c 


0.3 


VAR 99 


0.0 


0.0 


0.0 


0.0 


o.c 


0.0 


VAR 100 


0.0 


0.0 


0.0 


o.c 


O.G 


0.0 


VAR 101 


0.0 


0.0 


o.c 


0.0 


o.c 


0. C 


VAR 102 


0.0 


0.0 


0.0 


0.0 


o.c 


0.0 


VAR 103 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 104 


0.0 


0.0 


0.3 


0.0 


0.0 


0.0 


VAR 105 


0.0 


0.0 


0.0 


o.c 


0.0 


0.0 


VAR 106 


0.0 


o.c 


0.0 


0.0 


0.0 


0.0 


VAR 107 


0.0 


0.0 


0.0 


o.c 


o.c 


0.0 


VAR 103 


0.0 


0.3 


CO 


CO 


0.0 


0.3 


V«R 109 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 110 


0.0 


0.0 


0.0 


0.0 


o.c 


0.0 


VAR 111 


o.c 


o.c 


CO 


0.0 


o.c 


0.0 


VAR 112 


o.c 


CO 


CO 


CO 


0.0 


CO 


VAR 113 


0.0 


CO 


0.0 


0.0 


o.c 


0.0 


VAR 114 


o.c 


0.0 


CO 


o.c 


0.0 


0.0 


VAR 115 


0.0 


0.0 


0.0 


o.c 


o.c 


0.0 


VAR 116 


o.c 


0.0 


CO 


0.0 


CO 


CO 


VAR 117 


0.0 


o.c 


CO 


0 • c 


o.c 


0.0 


VAR 118 


o.c 


0.0 


0.0 


0.0 


0.0 


0.0 


VAR 119 


0.0 


0.0 


CO 


CO 


o.c 


0.0 


VAR 120 


0.C46 


0.025 


-0.0J4 


0.061 


-0.C64 


0.C50 


VAR 1*1 


u.OOn 


-0.045 


0.02U 


0.045 


-0.1.3C 


0.C41 



READING. 
O.C 
0.0 
0.0 
CO 
0.3 
0.0 
0.0 
0.0 ' 
0.0 
CO 
0.0 
0.0 
0.0 

o.o ■ 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

3.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.602 

0.440 

0.472 

0.335 

0.522 

0.612 

0.349 

0.220 

0.357 

0.46S 

0.644 

0.462 

0.439 

0.529 

0.372 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

CO 

0.0 

0.0 

0.165 

0.037 



M RITE -HC 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.3 
0.0 

0.0 

0.0 
0.0 

0.0 

0.0 

0.0 

0.0 

0.3 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.663 

0.374 

0.220 

0.254 

0.327 

0.366 
0.293 
0.319 
0.273 
0.5e9 
0.437 
0.632 
0.334 
0.456 
0.442 
0.122 
0.129 



ERIC 



Table 12 
Test of Two Competing Models 



a. Estimated True-Score Correlations between Three a Priori Constructs in 
Accordance with Test Design 

(Coefficient of determination = 0.147) 



Communication Skills 
General Knowledge 
Prof Knowledge 



Communication 
Skills 

1.000 

.871 

.927 



General 
Knowledge 



1.000 
.855 



Professional 
Knowledge 



1.000 



Estimated True-Score Correlations between Two Empirically Inferred 
Constructs 

(Coefficient of determination = 0.219) 



Verbal Math 
Verbal 1.000 
Mathematics .780 1.000 



Table 13 



Standardized Factor Loadings for Three-Factor Model as per Test Design" 

(N = 13,059) 





COMM-SKL 




















3SN-KNOW 


PROF-KN 










VAR 1 


0.0~ 


07533 


0.0 




VAR 


60 


0.3 


0.0 


0.642 


VAR 2 


0.0 


0.361 


0.0 




VAR 


61 


0.0 


0.0 


0.394 


VAR 3 


0.0 


0.606 


0.0 




VAR 


62 


0.0 


0.0 


0.499 


VAR 4 


0.0 


0.223 


0.0 




VAR 


63 


0.0 


0.0 


0.353 


VAR S 


0.0 


0.515 


0.0 




VAR 


64 


0.0 


0.0 


0.357 


VAR 6 


0.0 


0.402 


0.0 




VAR 


65 


0.0 


0.0 


0.404 


VAR 7 


0.0 


0.509 


0.0 




VAR 


66 


0.0 


0.0 


0.494 


VAR 8 


0.0 


0.467 


0.0 




VAR 


67 


0.0 


0.0 


0.614 


VAR 9 


0.0 


0.312 


0.0 




VAR 


68 


0.0 


0.0 


0.285 


VAR 10 


0.0 


C.437 


0.0 




VAR 


69 


0.0 


0.0 


0.131 


VAR 11 


0„0 


0.603 


0.0 




VAR 


70 


0.0 


0.0 


0.403 


VAR 12 


0.0 


0.572 


0.0 




VAR 


71 


0.0 


0.0 


0.391 


VAR 13 


0.0 


0.427 


0.0 




VAR 


72 


0.0 


0.0 


0.410 


VAR 14 


0.0 


0.434 


0.0 




VAR 


73 


0.0 


0.0 


0.339 


VAR 15 


0-.0 


0.326 


0.0 




VAR 


74 


0.580 


0.0 


0.0 


VAR 16 


0.0 


0.350 


0.0 




VAR 


7S 


0.170 


0.0 


0.0 


VAR 17 


0.0 


0.402 


0.0 




VAR 


76 


0.584 


0.0 


0.0 


VAR 18 


G.C 


0.261 


0.0 




VAR 


77 


0.462 


0.0 


0.0 


VAR 19 


0.0 


0.312 


0.0 




VAR 


78 


0.561 


0.0 


0.0 


VAR 20 


0.0 


0.175 


0.0 




VAR 


79 


0.527 


0.0 


0.0 


VAR 21 


0.0 


0.420 


0.0 




VAR 


30 


0.432 


0.0 


0.0 


VAR 22 


0.0 


0.246 


0.0 




VAR 


31 


0.297 


0.0 


0.0 


VAR 23 


0.0 


0.4o2 


0.0 




VAR 


32 


0.239 


0.0 


0.0 


VAR 24 


o.c 


0.333 


0.0 




VAR 


83 


0.459 


0.0 


0.0 


VAR 25 


0.0 


0.252 


0.0 




VAR 


84 


0.595 


0.0 


0.0 


VAR 26 


0.0 


0.382 


0.0 




VAR 


85 


0.122 


0.0 


3.0 


VAR 27 


0.0 


0.294 


0.0 




VAR 


86 


0.602 


0.0 


0.0 


VAR 28 


0.0 


0.401 


0.0 




VAR 


87 


0.503 


0.0 


0.0 


VAR 29 


0.0 


0.451 


0.0 




VAR 


88 


0.415 


0.0 


0.0 


VAR 30 


0.0 


0.200 


0.0 




VAR 


39 


0.453 


0.0 


0.0 


VAR 31 


0.0 


0.313 


0.0 




VAR 


90 


0.055 


0.0 


0.0 


VAR 32 


0.0 


0.375 


0.0 




VAR 


91 


0.535 


o.c 


0.0 


VAR 33 


0.0 


0.390 


0.0 




VAR 


92 


0.452 


0.0 


0.0 


VAR 34 


0.0 


0.399 


0.0 




VAR 


93 


0.222 


0.0 


0.0 


VAR 35 


0.0 


0.391 


0.0 




VAR 


94 


0.535 


0.0 


0.0 


VAR 36 


0.0 


0.520 


0.0 




VAR 


95 


0.571 


0.0 


0.0 


VAR 37 


0.0 


0.421 


0.0 




VAR 


96 


0.450 


0.0 


0.0 


VAR 33 


0.0 


0.4(36 


0.0 




VAR 


97 


0.311 


0.0 


0.0 


VAR 39 


0.0 


0.327 


0.0 




VAR 


98 


0.351 


0.0 


0.0 


VAR 40 


0.0 


0.209 


0.0 




VAR 


99 


0.495 


0.0 


0.0 


VAR 41 


0.0 


0.377 


0.0 




VAR 


100 


0.650 


0.0 


0.0 


VAR 42 


0.0 


0.412 


0.0 




VAR 


101 


0.490 


0.0 


0.0 


VAR 43 


0.0 


0.340 


0.0 




VAR 


102 


0.536 


0.0 


0.0 


VAR 44 


0.0 


0.251 


0.0 




VAR 


103 


0.560 


0.0 


0.0 


VAR 45 


0.0 


0.316 


0.0 




VAR 


104 


0.41 5 


0.0 


0.0 


VAR 46 


0.0 


C.253 


0-0 




VAR 


105 


0.502 


0.0 


0.0 


VAR 47 


0.0 


0.550 


r.o 




VAR 


106 


0.340 


0.0 


0.0 


VAR 43 


0.0 


0.213 


0.0 




VAR 


107 


0.247 


0.0 


0.0 


VAR 49 


0.0 


C.606 


0.0 




VAR 


103 


0.241 


0.0 


0.0 


VAR 50 


o.c 


0.449 


o.c 




VAR 


109 


0.239 


0.0 


0.0 


VAR 51 


0.0 


0.246 


0.0 




VAR 


110 


0.281 


0.0 


0.0 


VAR 52 


0.0 


0.423 


0.0 




VAR 


111 


0.230 


0.0 


0.0 


VAR 53 


0.0 


0.335 


0.0 




VAR 


112 


0.31o 


0.0 


0.0 


VAR 34 


0.0 


0.459 


0.0 




VAR 


113 


0.236 


0.0 


0.0 


va« 55 


0.0 


o.?oo 


0,0 




VAR 


114 


0.565 


0.0 


0.0 


VAR 56 


0.0 


-0.069 


0.0 




VAR 


115 


0.384 


0.0 


0.0 


VAR 57 


0.0 


0.463 


0.0 




VAR 


116 


0.565 


0.0 


0.0 


VAR 53 


0.0 


0.335 


0.0 




VAR 


117 


0.419 


0.0 


0.0 


VAR 59 


0.0 


0.342 


0.0 




VAR 
VAR 


113 
119 


0.440 
0.444 


3.0 

0.0 


0.0 
0.0 












VAR 


120 


0.162 


0.0 


0.0 










5-i 


VAR 


121 


0.136 


0.0 


0.0 



a 

ERIC 



'See Table 9 for translation of variable numbers to items in Core Battery. 



Table 14 



Standardized Factor Loadings for Two-Factor Model'" 

(X = 13,059) 





VER8AL 


MATH 


VAR 1 


0.54? 


0.0 


VAR 2 


0.376 


0.0 


VAR 3 


0.625 


0.0 


V AR 4 


0.229 


0.0 


VAR 5 


0.529 


0.0 


VAR 6 


0.413 


0.0 


VAR 7 


0.523 


0.0 


VAR 8 


0.479 


o.o 


VAR 9 


0.834 


0.0 


VAR 10 


0.501 


0.0 


VAR 11 


0.621 


0.0 


VAR 12 


0.586 


0.0 


VAR 13 


0.433 


0.0 


VAR 14 


0.445 


0.0 


VAR 15 


0.335 


0.0 


VAR 16 


0.0 


0.603 


VAR 17 


0.0 


0. 608 


VAR 1 3 


0.0 


0. 370 


VAR 19 


0.0 


0.497 


VAR 20 


0.0 


0. 246 


VAR 21 


0.0 


0.632 


VAR 22 


0.0 


0.391 


VAR 23 


0. 0 


0. 691 


VAR 24 


0.0 


0. 544 


VAR 25 


0.0 


0. 397 


VAR 26 


CO 


0.533 


VAR 27 


0.0 


0. 451 


VAR 28 


0.0 


0.584 


VAR 29 


0. 0 


0.596 


VAR 30 


0.0 


0.331 


VAR 31 


0.327 


0.0 


VAR 32 


0.337 


0.0 


VAR 33 


0.402 


0.0 


VAR 34 


0.407 


0.0 


VAR 35 


0.402 


0.0 


VAR 36 


0.536 


0.0 


VAR 37 


0.432 


0.0 


VAR 38 


0.500 


0.0 


VAR 39 


0.333 


0.0 


VAR 40 


0.217 


0.0 


VAR 41 


0.387 


0.0 


VAR 42 


0.424 


0.0 


VAR 43 


0.347 


0.0 


VAR 44 


0.255 


0.0 


VAR 45 


0.326 


0.0 


VAR 46 


0.266 


0.0 


VAR 47 


0.561 


0.0 


VAR 48 


0.221 


0.0 


VAR 49 


0.625 


0.0 


VAR 50 


0.463 


0.0 


VAR 51 


0.253 


0.0 


VAR 52 


0.442 


0.0 


VAR 53 


0.393 


0.0 


VAR 54 


0.471 


0.0 


VAR 55 


0.207 


0.0 


VAR 56 


-0.063 


0.0 


VAR 57 


0.475 


0.0 


VAR 58 


0.345 


0.0 


VAR 59 


0.350 


0.0 









KATH 


VAR 


60 


0.394 


0.0 


VAR 


61 


0.239 


0.0 


VAR 


62 


0.514 


0.0 


VAR 


63 


0.310 


0.0 


VAR 


64 


0.294 


0.0 


VAR 


65 


0.354 


0.0 


VAR 


66 


0.407 


0.0 


VAR 


67 


0.555 


0.0 


VAR 


63 


0.201 


0.0 


VAR 


69 


0.211 


0.0 


VAR 


70 


0.416 


0.0 


VAR 


71 


0.350 


0.0 


VAR 


72 


0.403 


0.0 


VAR 


73 


0.230 


0.0 


VAR 


74 


0.313 


0.0 


VAR 


75 


0.193 


0.0 


VAR 


76 


0.349 


0.0 


VAR 


77 


0.273 


0.0 


VAR 


73 


0.506 


0.0 


VAR 


79 


0.502 


0.0 


VAR 


30 


0.391 


0.0 


VAR 


31 


0.279 


0.0 


VAR 


32 


0.284 


0.0 


VAR 


33 


0.392 


0.0 


VAR 


84 


0.331 


0.0 


VAR 


85 


C-110 


0.0 


VAR 


86 


0.380 


0.0 


VAR 


87 


0.430 


0.0 


VAR 


88 


0.400 


0.0 


VAR 


89 


0.422 


0.0 


VAR 


90 


0.074 


0.0 


VAR 


91 


0.470 


0.0 


VAR 


92 


0.459 


0.0 


VAR 


93 


0.235 


0.0 


VAR 


94 


0.472 


0.0 


VAR 


95 


0.522 


0.0 


VAR 


96 


0.450 


0.0 


VAR 


97 


0.352 


0.0 


VAR 


93 


0.355 


0.0 


VAR 


99 


0.425 


0.0 


VAR 


100 


0.512 


0.0 


VAR 


101 


0.369 


0.0 


VAR 


102 


0.422 


0.0 


VAR 


103 


0.446 


0.0 


VAR 


104 


0.278 


0.0 


VAR 


105 


0.428 


0.0 


VAR 


106 


0.301 


0.0 


VAR 


107 


0.294 


0.0 


VAR 


108 


0.343 


0.0 


VAR 


109 


0.355 


0.0 


VAR 


110 


0.327 


0.0 


VAR 


111 


0.309 


0.0 


VAR 


112 


0.266 


0.0 


VAR 


113 


0.345 


0.0 


VAR 


114 


0.471 


0.0 


VAR 


115 


0.431 


0.0 


VAR 


116 


0.451 


0.0 


VAR 


117 


0.441 


0.0 


VAR 


118 


0.371 


0.0 


VAR 


119 


0.407 


0.0 


VAR 


120 


0.121 


0.096 


VAR 


121 


0.090 


0.045 



a 

ERIC 



JO 

^See Table 9 for translation of variable numbers to items in Core Battery 



Table 15 



Estimated True-Score Correlations for Low Readers 



(6) 









(N = 


3,265) 








(1) 


(2) 


(3) 


(4) (5) 


(1) 


Social Studies 


! LOO? 








(2) 


Mathematics 


| .463 


1.000 


"S 












N, 


General 


(3) 


Lit/Fine Arts 


! .570 


.530 


1.000" 




(4) 


Science 


J .738 


.674 


.677 


i.oocT^ 


(5) 


Prof Knowledge 


.549 


.593 


.678 


.735 1.000 


(6) 


Listening 


.518 


.589 


.673 


.688 .809 


(7) 


Reading 


-.069 


.112 


.163 


-.094 .063 


(S) 


Writing (m/c) 


.439 


.543 


.595 


.642 .765 



(7) (8) 



Communication 
/ Skills 



.235 1.000 " 



,158 1.000"- 



'Dotted lines enclose correlations that should be high compared with 
correlations outside of the dotted lines in order for the test to be valid as a 
measure of three constructs: Communication Skills, General Knowledge, and 
Professional Knowledge. High correlations inside the lines would indicate 
convergent validity; low correlations outside the lines would indicate 
discriminant validity. 



Appendix A 

Intercorrelations of Variables Analyzed, by Subgroup 

Definitions of Variables in Matrices 

READ-1 Number of even-numbered items answered right in Reading test 

READ-2 Number of odd-numbered items answered right in Reading test 

LIST-1 Number of even-numbered items answered right in Listening test 

LIST-2 Number of odd-numbered items answered right in Listening test 

WRITE-1 Number of even-numbered items answered right in multiple-choice 
writing test 

WRITE- 2 Number of odd-numbered items answered right in multiple-choice 
writing test 

LITART-1 Number of even-numbered items answered right in Literature and 
Fine Arts test 

LITART-2 Number of odd-numbered items answered right in Literature and 
Fine Arts test 

SCI-1 Number of even-numbered items answered right in Science test 
SCI-2 Number of odd-numbered items answered right in Science test 
SOCSTD-1 Number of even-numbered items answered right in Social Studies test 
SOCSTD-2 Number of odd-numbered items answered right in Social Studies test 
PROFKN-1 Number of items answered right in first Professional Knowledge test 
PROFKN-2 Number of items ansv/ered right in second Professional Knowledge test 
PROFKN-3 Number of items ansv/ered right in third Professional Knowledge test 
ESSAY- 1 Essay score assigned by first rater 
ESSAY-2 Essay score assigned by second rater 

MATH-1 Number of even-numbered items answered right in Mathematics test 
rIATH-2 Number of odd-numbered items ansv/ered right in rlathenatics test 
GPA Self-reported grade-point average 



a 

ERIC 



A-l 



Correlation Matrix for White Males 
N = 791 



COKRELAT ION MATRIX TU flE ANAL YZED 



> 
I 

to 



REAO-1 

REAO-2 

L I ST- 1 

LIST-2 

HRI Tt-1 

HRtTfc-2 

L 1 TART- 1 

LI IAKT-2 

SU-1 

SCI-2 

SUCSTU-1 

SOCSTD-2 

PKOFKN- 1 

PROFKN-2 

PRUFKN-3 

ESSAr-1 

ESSAY-2 

MATH- 1 

MATH-2 

CPA 



ttEAQ:l_- 
1 .QUO 
0.693 
0.637 
0.63 J 
0.634 
0.64? 
0.639 
0.624 
0.599 
0.641 
0.610 
0.664 
0.661 
0.675 
0.651 
0.426 
0.418 
0.531 
0.551 
0.304 



BEAQr2 

1 .000 
0.61 7 
0.591 
0. 596 
0.630 
0.649 
0.616 
0.567 
0.628 
0.61 1 
0.641 
0.667 
0.657 
0.639 
0. 379 
0. 391 
0.514 
0.535 
0.299 



L1SI = 1 — 










UIftBT-2 




SCl=2_. . 


1 .000 
















0.696 


1.000 














0.549 


0.562 


1.000 












0.549 


0.564 


0.730 


1 .000 










0.598 


0.603 


0.617 


0.625 


1 .000 








0. 569 


0.5R7 


0. 608 


0.618 


0.734 


1.000 






0.524 


0.521 


0.493 


0.519 


0.574 


0.541 


1.000 




0.554 


0.580 


0.575 


0.591 


0.672 


0.617 


0.665 


I. 000 


0.563 


0.597 


0.543 


0.567 


0.672 


0.629 


0.586 


0.667 


0.590 


0.605 


0.578 


0.600 


0.661 


0.615 


0.570 


0.639 


0.606 


0.623 


0.604 


0.630 


0.693 


0.634 


0.582 


0.666 


0.583 


0.589 


0.587 


0.606 


0.673 


0.613 


0.554 


0.680 


0.554 


0.568 


0.577 


0.601 


0.635 


0.605 


0.572 


0.662 


0. J68 


0.370 


0.479 


0.445 


0.410 


0.399 


0.316 


0.352 


0. 386 


0.357 


0.472 


0.445 


0.436 


0.430 


0.297 


0.355 


0.513 


0.505 


0.506 


0.493 


0.530 


0.51 3 


0.601 


0.608 


0.503 


0.484 


0.501 


0.501 


0.485 


0.505 


0.553 


0.564 


0.304 


0.262 


0.314 


0.326 


0.323 


0.31 7 


0.308 


0.309 
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CORRELATION MATRIX TO flE ANALYZED 





SQCilUrl. 


SU0.SIU:2 




E8QEKH;2 


SOCSTD-1 


1 .000 








SOCSTO-2 


0.711 


1.000 






PKUFKN-l 


0.648 


0.659 


1 .000 




PRUFKN-2 


0.651 


0.656 


0. 762 


1.000 


PROf KN-3 


0.641 


0.629 


0. 778 


0. 782 


tSSAY-l 


0.359 


0.389 


0.407 


0.420 


ESSAr-2 


0.353 


0.410 


0.406 


0.41 1 


MATH- 1 


0.538 


0.53J 


0.529 


0.477 


MATH-2 


0.531 


0.523 


0.485 


0.468 


OPA 


0.312 


0.3*4 


0.358 


0.369 



EBQEHNrJ ESSAY=1- ESSAX-2_ MftlUrl-- MftlHr2. 



1.000 
0.402 
0.405 
0.510 
0.491 
0.369 



1 .000 
0.578 
0.317 
0.299 
0.238 



1.000 
0.290 
0.301 
0.218 



1.000 
0.685 
0.237 



1.000 
0.199 



GEA 



I. 000 



S8 



5y 



Correlation Matrix for White Females 
N = 3,797 



tUKKI LA f 1 ON 


MA1K1X TO BE 


ANAL Y/EO 










BEflU-2 


LUI:1- 


LlSI=2-_ 




HE AL>-1 


1 .000 










KtAO-2 


0.64H 


I # 000 








I IST-1 


0.6H 


0. 55'* 


1 .000 






L IS1-2 


0.60'J 


0.569 


0.654 


1.000 


1 .000 


WR1 Tt-1 


0.576 


0.541 


0.534 


0.52 5 


WRI TE-2 


0.5M1 


0..546 


0.541 


0.538 


0.715 


LHAKT-1 


D.601 


0. 584 


0.564 


0.550 


0.588 


L ITART-2 


0.550 


0. 530 


0. 502 


0.491 


0.548 


SCI-1 


0.496 


0. 46 \ 


0.441 


0.438 


0.444 


SCI-2 


0.580 


0. 548 


0.518 


0.521 


0.533 


SOCSTD-1 


0.556 


0.533 


0. 520 


0. 506 


0.506 


SUCSTD-2 


0.568 


0.53M 


0.518 


0.521 


0.529 


PROFKN-1 


0.61T 


0.603 


0.565 


0.553 


0.551 


PROFKN-2 


0.602 


0.580 


0.528 


0.529 


0.531 


PROFKN-3 


0.60? 


0.593 


0.550 


0.550 


0.543 


kSSAY-1 


0.295 


0.7B2 


0.261 


0.274 


0.324 


ESSAY-2 


0.288 


0.262 


0.263 


0.265 


0.334 


MATH- 1 


0.541 


0.476 


0.489 


0.499 


0.480 


MAFH-2 


0.531 


0.454 


0.466 


0.469 


0.486 


CPA 


0.35J 


0.314 


0.298 


0.289 


0.314 


CORRELATION 


MATRIX TO BE 


ANALYZED 








SUCSIQ-1 


SQCS.IU=2 






eSUEKN-1 


SClCSTD-1 


1 .000 










SUCSTD-2 


0.624 


1.000 








PRUFKN- 1 


0.56H 


0.588 


1.000 






PROFKN-2 


0.557 


0.573 


0.703 


1.000 




PROFKN-3 


0.579 


0.588 


0. 709 


0.715 


1.000 


ESSAV-l 


0.246 


0.267 


0.281 


0.262 


0.255 


ESSAY-2 


0.261 


0.257 


0.295 


0.269 


0.274 


MATH- 1 


0.497 


0.484 


0.482 


0.482 


0.497 


MAFH-7 


0.473 


0.467 


0.465 


0.455 


0.472 


UPA 


0.311 


0. 322 


0.371 


0.369 


0.393 



M81IEr2_ 



1.000 
0.611 
0.567 
0.470 
0.557 
0.528 
0.545 
0.576 
0.556 
0.565 
0.349 
0.330 
0.502 
0.503 
0.324 



LIISBTrl tlIftBI;2 SClrl.-.. SCItZ. 



1.000 
0.492 
0.226 
0.752 
0.186 



1.000 
0.666 
0.516 
0.603 
0.590 
0.608 
0.618 
0.590 
0.5<>8 
0.314 
0.302 
0.490 
0.45T 
0.325 



ESSAXrl. ESSAI=2. 



1 .000 
0.203 
0.236 
0.145 



1.000 
0.477 
0.538 
0.525 
0.549 
0.561 
0.541 
0.525 
0.289 
0.292 
0.458 
0.428 
0.304 



1 .000 
0.652 
0.289 



1.000 
0.571 
0.503 
0.493 
0.498 
0.503 
0.520 
0.211 
0.230 
0.475 
0.458 
0.787 



H6IH=1._ MAIH = 2. 



1.000 
0.579 
0.577 
0.589 
0.592 
0.602 
0.257 
0.259 
0.554 
0.SI1 
0.331 



1.000 
0.272 



1.000 
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Correlation Matrix for Black Males 
N = 97 



CORkELATION MATRIX TO BE ANALVZEO 



> 



REAO-1 

READ-2 

UST-l 

LIST-2 

WRITE-1 

WRITE-2 

L i TAR T- 1 

llfART-2 

SCI-1 

SCI -2 

SGCSTO-l 

SUCSTO-2 

PRUFKN- 1 

PROFKN-2 

PROFKN-3 

ESSAV-1 

ESSAV-2 

HAIH-t 

MATH-2 

CPA 



1.000 
0.561 
0.561 
0.516 
0.57J 
0. 355 
0.439 
0.516 
0.512 
0.419 
0.431 
0.655 
0.524 
0.449 
0.543 
0.30? 
0.337 
0.399 
0.463 
0.239 



R£AQ;2__ USIrl- L1SI=2__ «RIIE=i. MBIIE=2- LIIaBIrl UUBI=2 SCIrl. 



SCI-2-... 



1.000 
0.607 
0.489 
0. 399 
0.357 
0.497 
0.570 
0.497 
0.337 
0.523 
0.521 
0.476 
0.582 
0.466 
0.222 
0.232 
0.432 
0.423 
0.200 



1.000 
0.575 
0.454 
0.492 
0.428 
0.539 
0.497 
0.527 
0.510 
0.535 
0.488 
0.531 
0.567 
0. 188 
0.247 
0.51 7 
0.533 
0. 193 



SOCSTD-1 

SOCSTO-2 

PROFKN-1 

PKOf-KN-2 

PROFKN-3 

ESSAV-l 

ESSAV-2 

HATH- 1 

MATH-2 

GPA 



CORRELATION MATRIX TO BE ANALVZEO 



1.000 
0.591 
0.551 
0.588 
0.480 
0.280 
0.21 7 
0.401 
0.434 
0. 1 79 



1 .000 
0.627 
0.5R6 
0.591 
0. 1 72 
0.205 
0.397 
0.440 
0. 162 



1.000 
0.641 
0.685 
0.270 
0.279 
0.251 
0.347 
0. 160 



1.000 
0.416 
0.466 
0.461 
0.49 7 
0.497 
0.459 
0.541 
0.501 
0.434 
0.427 
0.429 
0.123 
0. 172 
0.437 
0.436 
0.305 



1.000 
0.661 
0.346 
0.361 
0.401 
0.469 
0. 145 



1.000 
0.625 
0.445 
0.5 72 
0.352 
0.487 
0.442 
0.510 
0.51 1 
0.531 
0.550 
0.293 
0.357 
0.408 
0. 548 
0.278 



1.000 
0.38 3 
0.470 
0.282 
0.406 
0.409 
0. 392 
0.4R4 
0. 390 
0.41 5 
0.245 
0.412 
0.264 
0.402 
0. 31 3 



1.000 








0.642 


1.000 






0.400 


0.473 


1.000 




0.504 


0.491 


0.480 


1.000 


0.530 


0.467 


0.512 


0.375 


0.536 


0.551 


0.561 


0.439 


0.486 


0.632 


0.507 


0.368 


0.460 


0.608 


0.609 


0.450 


0.562 


0.58? 


0.529 


0.49? 


0.112 


0.200 


0.259 


0.125 


0.248 


0.740 


0. 186 


0.223 


0.315 


0.429 


0.454 


0.457 


0.326 


0.474 


0.511 


0.469 


0.197 


0.724 


0.128 


0.264 



EBDEKd-2 EBQEKCfrJ ESSay=l. ESSfly-2_ Baldrl.. SAIH=2„ 



1.000 
0.232 
0.346 
0. 323 
0.382 
0. 145 



1 .000 
0.640 
0.29 1 
0.776 
0. 153 



1 .000 
0.157 
0.258 
0. 190 



1.000 
0.674 
0.134 



GPA... 



1.000 
0.202 



1.000 
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BEST COPY AVAILABLE 



Correlation Matrix for Black Females 
N = 498 



CDKKr l A 7 I ON MATKIX TIJ IIF ANIUY/fl) 





Ht&U-i-- 


KEAU-£_- 


LiSL=l_. 










L1I»BI=2 




SCl-2 


RfcAO-1 


1 .000 




















HCAU-2 


0.604 


1 .000 


















LISI-1 


0.606 


0 . 564 


1 .000 
















LIST-? 


0.5H2 


0 . 564 


0.6// 


1 .000 














wri rc-i 


0.602 


0. 530 


0.520 


0.548 


1.000 












wri rt-2 


0.5 70 


0.509 


0 . 504 


0.493 


0.641 


1.000 










L ITARl-l 


0.613 


0.559 


0.536 


0.600 


0 5 60 


0.50? 


1.000 








L I TART-2 


0.584 


0 • 52 H 


0.48/ 


0.532 


0.5 35 


0.535 


0.623 


1 • 00 0 






SC1-1 


0.46? 


0.456 


0.460 


0.481 


0.4 74 


0.406 


n Kr\ "y 
u . 7U c 


0 • A 


1.000 




SC1-2 


0.50/ 


0.455 


0.457 




0.466 


0.410 


0.508 


0 • **** f> 


0.4 76 


1 . 000 


SOCSIO- 1 


0.552 


0.524 


0.508 


0.510 


0.504 


0.434 


0.579 


0*513 


0.468 


n Gil 
U . T> 1 1 


socsin-2 


0.583 




0.514 








0.587 


n ^7 a 

U • JC. 0 


0.493 


0.498 


PRUI-KN-1 


0.621 


0 .606 


0.589 


0 . 60 3 


0 . 586 


0 . 558 


0 . 590 


0 • 58 9 


0. 483 


0. 516 


PROHKN-2 


0.645 


0.610 


0.583 


0.625 


0.601 


0.554 


0.591 


0.571 


0.443 


0.507 


PROKKN-3 


'1.606 


0.586 


0.56 3 


0.556 


0.552 


0 .508 


0.534 


0.501 


0.418 


0.523 


CSSAY-1 


0.482 


0.4/4 


0. 391 


0.396 


0.446 


0 .393 


0.367 


0. 386 


0.305 


0.773 


essay-? 


0.436 


0. 385 


0. 304 


0.349 


0.368 


0.356 


0.345 


0.385 


0.261 


0.27? 


MATH-1 


0.450 


0.348 


0.398 


0.412 


0.406 


0.376 


0. 371 


0.385 


0.364 


0.377 


MAIH-2 


0.42H 


0. 366 


0.407 


0.435 


0.445 


0.3 72 


0.438 


0.425 


0.346 


0.411 


GPA 


0.202 


0.220 


0.155 


0.1/9 


0.249 


0.239 


0.194 


0.229 


0.117 


0. 181 


CORRfcLAT ION 


MATRIX 10 BE 


ANALYZED 




















SU.CSI.Uz2 


EB.UC.KNrl. 


EB.QE.tLN:; 2. 


EBUEKN;3 


ESSftXrl.- 


ESSAY. z2. 




HaIH r 2.._ 


ces 


SUL-STD-l 


1 .000 




















sucsin -2 


0.56b 


1 .000 


















PROFKN- 1 


0.536 


0. 566 


1 .000 
















PRO! KN-2 


0.031 


0.506 


0.718 


1.000 














PRlirKN-3 


0.51/ 


D.559 


0.711 


0.694 


1.000 












ESSAY-1 


0. 320 


0.416 


0.465 


0.456 


0.433 


1 .000 










TSSAY-2 


0.271 


0.318 


0. 364 


0.391 


0.3/8 


0 .587 


1 .000 








MAIH-1 


0. 36/ 


0. 341 


0.423 


0.404 


0. 355 


0.279 


0.20 1 


1 .000 






MAIH-2 


0. 398 


0.393 


0.446 


0.404 


0. 363 


0. 307 


0.290 


0.476 


1.000 




GPA 


0. 160 


0. 177 


0.25* 


0.243 


0.298 


0. 189 


0.148 


0.08 7 


0.161 


1.000 
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Appendix B: Tetrachoric Correlations Among Items (N = 13,059) 

GKSS1 GKSS 3 GKSS 5 GKSS 7 3KSS9 GKst 1 1 GKSS 1 3 GKSS 1 5 GKS ? S17 G*M19 mIw! Eislzj 



1 GKSS 1 1.000 

2 GKSS 3 0.104 1.000 

3 GKSS 5 0.186 0.228 1.000 

4 GKSS 7 0.030 0.086 0.098 1.000 



I «?; S:!H 8:iH S:S!S 5:!!? , ioo 

? SIUl! S:!« ..... !•»• »■»" !•"< • 



• asi! s:;si ::ia ^ wii t-IH SiiS tlH ills '.-.as ..... 

ii SKI! £ I : : : : ..». S:HS ..... 



11 GKSS tl u.ifj -•;«' n ? „ 0 2 . 7 0 . 266 o.2b2 0.386 0.250 u„ ~iu i.uuu 

ii as g fcU : : : : « o- ..,„ o.„o ..m 

?i KKS S: 1 oll 5:1.. o 0 :^ : : : s : »° g:lis 8:?K S:^ 

i* «" 1 °- 101 2*li;i * f? o S: ° : o o 2:202 S: u ; 0.230 0.329 

17 GKH 3 0.132 0.184 0.383 0. 37 0.055 0.219 u.ivo 0<Q9 ^ 0<1JJ 

5 °' 072 2-211 n* 1 ,tt 0 • U S'S« S: 42 0 ?36 0 1M 0.1 86 0.151 0.1 59 0.245 

1» GKM 6 0. 081 0.142 0.241 0.1 36 0.068 U.142 u. 0<10J Q>07 , 

20 GKH 8 0.052 0.025 0.030 0.076 0.052 0.053 0.079 U.uo< Q 6 Q<295 

21 GKH 10 0.114 0.141 0.272 0.122 0.099 0. 63 0.138 0 4* o<ij? Q> , 6 , 

22 «" 11 °-"8 0.1 02 0.198 0.078 0.0 7 0.121 0.0 J8 0 89 Q<169 q ^ 

23 GKH 13 0.158 0.1 21 0.232 0. 79 0.211 0.208 U.167 Q<175 Q<; , M 
2* «" 15 0.118 0.1 23 0.209 0. 42 0.143 0. 53 0 33 0 1 97 „.,„ 0 ., 15 
25 GKH 16 0. 075 0.1 55 0.262 0. 00 0.057 U.144 u.iuy u Q<2(H Q<2W 
2« GKH 18 0.120 0.118 0.260 0. 59 0.125 0. 48 0 29 0.1 V Q<151 Q<212 
27 GKH 20 0.091 0.097 0.194 0. 23 0.088 0.113 0 0 U 66 Q<216 Q<26i 

I 28 GKH 21 0.100 0.161 0.228 0. 61 0.11 2 0.206 0 41 0i2 „ 0 . 2 , 6 

M 29 GKH 23 0. 128 0.141 0.253 0.165 0.1 32 0.199 0. J7B Q<o35 0<0<)J Q<u9 

30 GKH 2< 0. 067 0.081 0.164 0.080 0.0 0.037 0. 074 Q<1M Q<165 

31 GKL 1 0.146 0.124 0.250 0.066 0.11 0.100 U.123 u.io Q<169 Q<187 

32 GKL 3 0. 096 0.1 39 0.248 0.050 0.1 21 0. 23 0. 59 0 204 0 233 , Q ^ r) 

33 GKL 7 0.094 0.137 0.253 0.052 0.094 0.148 U.15U u.ty Q5 0<1u7 

34 GKL 8 0.107 0.131 0.178 0.073 0. 63 0. 46 0. 57 0. 59 0 133 

J5 GKL 11 0.125 0.11 5 0.223 0.059 0. 60 0. 23 0. 38 0 63 Q<267 

B St !i 2:152 !: S '• : : : !:U! MS 

S Sit !? !:SU !:!!! S:?;? : : : ...» ■■»> ■:>» !:!U 

5 K S £«! !:!!! !:SI 8:K :S : '• : S I?! S5K !:!S 

42 GKL 28 0.144 0.1 36 0.21 2 0.040 0. 02 0. 27 0. 30 0. 93 0 211 Q<u5 

43 GKL 31 0. 102 0.091 0.170 0.052 0.1 24 0.113 0. 1 62 0. 20 0.1 *8 Q<126 

44 GKL 33 0.102 0.071 0.170 0.053 0.072 0.083 0.064 0. 20 0 10 0.153 

45 GKS 1 0. 090 0.11 2 0.209 0.066 0.107 0.099 0.105 0. 55 0 68 U.li Q<u? 

46 GKS 3 0. 091 0.090 0.189 0.089 0.100 0.104 0.076 0.1 71 0.164 Q<J17 

*' W* 5 °' U5 °' 17 * 2i 8 »2 0 09 08 0l047 o'.04 ^088 0.1 29 0.1 16 

48 GKS 7 0.076 0.053 0.079 0.068 0.091 °'?° 3 " ""?£ „ n ,, 4 n.2 6 1 0 . 3 2 0.365 

*9 «S 9 0.1B8 0. 225 0.456 0.1 04 0.228 0. 251 0. 245 0. 355 0. 35 4 0 261 U. 

0 "I 11 0.154 0.127 0.288 0.100 0.205 0.167 0.1 »6 0. 07 8 0.1 79 ^ 

51 GKS 1 3 0. 109 0.125 0.218 0.080 0.093 0.090 U.UO u. 0<271 

52 GKS 1 5 0.143 0.148 0.235 0.091 0.1 82 0.240 0. 93 0.270 u.«J Q<22; . 

53 MS 1 7 0. 077 0.111 0.264 0.075 0. 36 0.1 70 0. 59 0.J59 0 2 7 

54 GKS 1 9 0. 136 0. 173 0.301 0.102 0.173 0.207 0.1 74 . 0<1Q(S 

55 GKS 21 0.068 0.019 0.078 0.092 0.06 O.OSfi 0.071 0.051 _ _0 074 ^.^ 

56 SKS 23 -0.008 -0.007 -0.043 -0.004 -0.015 -0.055 0.022 0.067 U 0<27Q 0<21U 

57 MS 25 0.132 0.1 59 0.324 0.091 0. 74 0. 94 0. 02 0 304 „., 51 0 . 1 50 0.193 

58 GKS 27 0.105 0.1 00 0.218 0.093 0.157 0.121 0.115 u 4. Q<1 86 0<1()|) 
S« GKS 29 0.117 0.0)5 0.251 0.011 0.133 0.141 U.U91 ... Q isj n.190 0.2U4 
40 PK 1 0.125 0.179 0.300 0.084 0.036 0.135 0. 14 0. 42 0.165 0 155 0 

6 " 3 0. 037 0.1 24 0.254 0.041 0.042 0.064 0.050 0. 48 0. ,0 0 0.0*0 ^ ^ 

?r « p /k J ' °o: 1 o«7 S: 0 .i S^SS S:S.? S:SJS S:IS o:^° S:^ 5.«o 0.1,2 0.1,6 0.139 
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I 


•5 
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SSI 


SS3 


sss 




ft 


11 


0. 100 


0.160 


0.251 . 


45 


ft 


1 J 


0.103 


0.172 


0.2)6 


44 


ft 


1 4 


0. 120 


0.1)7 


0.336 


47 


ft 


1 8 


0. 1 53 


0.210 


0.407 




ft 


21 


0. 056 


0.078 


0.159 


♦ 9 


ft 


7 « 


0* 074 


0.097 


O.104 


70 


ft 


c o 


0* 108 


0.155 


0.242 


71 


ft 


7 II 


0* 12 5 


0.099 


0.270 


*7 

r « 


•w 

m 


3 I 


0. 120 


0.1 27 


0.260 


f J 


ft 


55 


0. 089 


0.092 


0.177 


71 
f % 


CS L 


1 


0. 078 


0.056 


0.198 


11 

f J 


CSL 


j 


0. 029 


0.064 


0.159 


1 • 


ft* 


4 


0. 11 5 


0.145 


0.519 


77 

f I 


est. 


4 


0. 079 


0.107 


0.256 


74 




1 1 


0. 173 


0.189 


0.566 


79 


CSL 


1 3 


0. 128 


0.150 


O.540 


10 


CSL 


1 6 


0. 124 


0.126 


0.229 


H 


CSL 


1 S 


0.064 


0.121 


0.220 


12 


CSL 


21 


0. 094 


0.102 


0.171 


|3 


CSL 


23 


0. 110 


0.149 


0.264 


|4 


CSL 


26 


0.129 


0.166 


0.528 


|j 


CSL 


26 


0.054 


0.052 


0.052 


16 


CSL 


J1 


0. 094 


0.181 


0.500 


17 


CSL 


35 


0. 145 


0.155 


0.542 


IS 


CSL 


36 


0. 100 


0.142 


0.318 


19 


CSL 


38 


0.106 


0.145 


0.515 


to 


CS* 


1 


G.038 


0.052 


0.038 


fl 


est 


5 


0.158 


0.186 


0.381 


f 2 


cs> 


5 


0.120 


0.149 


0.309 


f J 


csa 
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0.207 


0.406 
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0.124 


0.540 
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0.149 


0.517 
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0.090 


0.105 


0.211 
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0.139 


0.147 


0.558 
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0.095 


0.085 


0.214 
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0.091 


0.176 
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0.117 


0.111 


0.157 
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0.092 


0.201 
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0.101 


0.168 
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19 


0.091 


0.072 


0.166 
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0.097 


0.080 


0.149 


115 
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0.124 


0.110 


0.154 


114 
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28 


0.151 


0.197 


0.314 
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51 


0.114 


0.163 


0.293 
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0.160 


0.176 


0.364 
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37 


0.140 


0.131 


0.272 
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*0 


0.113 


0.154 


0.221 
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0.089 


0.147 


0.226 
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SS7 
0.023 
0.031 
0.091 
0.091 
0.005 
0.068 
0.078 
0.117 
0.095 
0.013 
0.081 
0.056 
0.152 
0.052 
0.092 
0.092 
0.078 
0.076 
0.026 
0.084 
0.066 
0.044 
0.098 
0.345 
0.092 
0.088 
-0.006 
0.078 
0.100 
0.035 
0.106 
0.092 
0.056 
0.064 
0.058 
0.053 
0.111 
0.066 
0.060 
0.120 
0.066 
0.078 
0.075 
0.050 
0.056 
0.070 
0.018 
0.060 
0.077 
0.070 
0.089 
0.072 
0.048 
0.079 
0.039 
0.060 



5 
SS9 
0.110 
0.071 
0.098 
0.181 
0.060 
0.061 
0.113 
0.112 
0.147 
0.068 
0.074 
0.067 
11.049 
0.061 
0.162 
0.134 
0.111 
0.080 
0.091 
0.109 
0.047 
0.028 
0.055 
0.152 
0.125 
0.150 
-0.015 
0.161 
0.16S 
0.085 
0.160 
0.169 
0.134 
0.141 
0.120 
0.097 
0.135 
0.085 
0.095 
0.101 
0.0/5 
0.071 
0.075 
0.108 
0.097 
0.120 
0.090 
0.070 
0.033 
0.105 
0.114 
0.150 
0.117 
0.137 
0.071 
0.073 
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0.099 

0.162 

0.252 

0.076 

0.091 

0.1 58 

0.157 

0.100 

0.064 

0.113 

0.050 

0.131 

0.096 

0.190 

0.177 

0.153 
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0.159 

0. 171 
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0.099 
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0.165 

0.055 

0.142 

0.107 

0.116 

0.082 

0.113 

0.077 
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0.092 

0.110 

0.165 
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0.156 

0.134 

0.117 

0.079 
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0.114 

0.128 

0.182 

0.095 

0.103 

0.147 

0.109 

0.116 

0.063 

0.094 
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0.159 

0.086 
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0.153 
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0.088 
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0.046 
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0.162 
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0.148 
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0.175 
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0.110 
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0.139 

0.084 

0.103 

0.069 

0.112 
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0.124 

0.124 
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SS15 
0.141 
0.172 
0.222 
0.550 
0.065 
0.088 
0.178 
0.163 
0.180 
0.129 
0.196 
0.107 
0.265 
0.196 
0.277 
0.235 
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0.178 
0.125 
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0.245 
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0.212 
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0.239 
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0.090 
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0.149 
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0.266 
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0.111 
0.223 
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0.226 
0.201 
0.209 
0.077 
0.157 
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0.103 
0.116 
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0.193 
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0.248 
0.231 
0.21 0 
0.191 
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0.180 
0.233 
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0.21 1 
0.148 
0.21 5 
0.198 
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0.155 
0.165 
0.136 
0.213 
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0.239 
0.170 
0.256 
0.197 
0.182 
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SS19 
0.105 
0.1)6 
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0.226 
0.080 
0.122 
0.150 
0.120 
0.150 
0.099 
0.103 
0.083 
0.184 
0.104 
0.195 
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0.138 
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0.120 
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0.147 
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0.108 
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0.122 
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0.172 
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0.144 
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0.145 
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0.296 
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0.224 
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0.144 
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0.140 
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0.143 
0. "12 
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0.195 
0.210 
0.132 
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0.289 
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0.218 
0.201 
0.208 
0.095 
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0.077 
0.201 
0.168 
0.247 
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0.164 
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0.125 
0.225 
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0.216 
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0.235 
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0.264 
0.244 
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0.254 
0.320 
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0.325 
0.214 
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0.262 
0.162 
0.200 
0.146 
0.150 
0.142 
0.149 
0.109 
0.09 5 
0.126 
0.145 
0.227 
0.187 
0.226 
0.210 
0.192 
0.174 









13 


14 


15 


16 








GKSS 25 GKSS 27 GKSS 29 GKH 1 


13 


GKSS 


25 


1. 000 








1* 


GKSS 


27 


0. 1 66 


1 .000 






1 5 


GUSS 


29 


o. 1 4 1 ; 


1 . 1 1 


1 .ann 




1 1> 


t*KH 


1 


0. 161 


0.116 


O.Wt 


1 .JuJ 


1 7 


GKH 


3 


0. 240 


0.177 


0.252 


0.278 


1 8 


GKH 


5 


0.132 


0.118 


0.058 


0.210 


1 9 


GKH 


6 


0.178 


0.128 


0.158 


0.246 


20 


GKH 


8 


0. 084 


0.085 


0.029 


0.093 


2 1 


GK H 


1 U 


0. 207 


0.201 


0.200 


0.330 


22 


GKH 


1 1 


0.125 


0.119 


0.190 


0.180 


21 


GKH 


1 3 


0.215 


0.196 


0.152 


0.397 


24 


GKH 


1 5 


0.179 


0.159 


0.112 


0.291 


25 


GKH 


1 6 


0.168 


0.120 


0.181 


0.153 


26 


GK H 


1 0 


0.189 


0. 161 


0.179 


0.268 


2 7 


GKH 


2 0 


0.121 


0.118 


0.090 


0.253 


28 


GKH 


21 


0. 233 


0.190 


0.143 


0.287 


29 


GKH 


23 


0.242 


0.216 


0.160 


0.269 


30 


GKH 


25 


0.119 


0.120 


0.159 


0.167 


31 


GKL 


1 


0. 1 21 


0.121 


0.1 31 


0.117 


32 


GKL 


3 


0. 1 36 


0.1 28 


0.148 


0.1 71 


33 


GKL 


7 


0. 188 


0.158 


0.222 


0.140 


34 


GKL 


8 


0. 1 42 


0.158 


0.148 


0.1 46 


35 


GKL 


1 1 


0. 118 


0.1 14 


0.110 


0.1 32 


36 


GKL 


13 


0.193 


0.158 


0.1 75 


0.188 


37 


GKL 


16 


0.172 


0. 143 


0.167 


0.1 39 


38 


GKL 


18 


0.192 


0.170 


0.169 


0.179 


39 


GKL 


21 


0.137 


0.1 04 


0.120 


0.14S 


40 


GKL 


23 


0. 054 


0.064 


0.051 


0.091 


41 


GKL 


26 


0.156 


0.146 


0.142 


0.1 32 


42 


GKL 


2 8 


0.141 


0.105 


0.220 


0.118 


43 


GKL 


31 


0. 100 


0.125 


0.110 


0.075 


44 


GKL 


3 3 


0.111 


0.095 


0.1 97 


0.1 17 


45 


GKS 


« 

i 


0. 150 


0.085 


0.132 


0.132 


46 


GK S 


3 


0.127 


0.110 


0.131 


0.1 26 


47 


GK S 


5 


0.232 


0.227 


0.194 


0.227 


48 


GKS 


7 


0. 073 


0.091 


0.057 


0.116 


49 


GKS 


9 


0.291 


0.246 


0.274 


0.253 


50 


GKS 


1 1 


0. 231 


0.212 


0.160 


0.1 89 


51 


GKS 


1 3 


0. 110 


0.074 


0.U5 


0.112 


52 


GKS 


1 5 


0. 197 


0.222 


0.168 


0.127 


53 


GK S 


1 7 


0. 15 3 


0.169 


0.142 


0.163 


54 


GKS 


1 9 


0. 201 


0.196 


0.130 


0.168 


5 5 


GKS 


tl 


0. 097 


0.098 


0.067 


0.145 


56 


GKS 


23 


-0. 011 


-0.054 


0.030 


-0.028 


57 


GKS 


25 


0. 220 


0.191 


0.203 


0.167 


58 


GKS 


2 7 


0. 160 


0.124 


0.113 


0.143 


CO 

5V 


GKS 


"i a 
C V 


0. 170 


0.119 


0.140 


0.181 


60 


PK 


1 


0.179 


0.122 


0.218 


0.144 


61 


P K 


3 


0.081 


0.045 


0.123 


0.121 


62 


PK 


6 


0.212 


0.177 


0.1 65 


0.191 


63 


PK 


8 


0. 155 


0.063 


0.167 


0.1 29 


64 


P K 


1 1 


0. 142 


0.083 


0.162 


0.097 


A C 

0) 


D If 


1 3 


0.155 


0.096 


0.118 


0.134 


66 


PR 


16 


0.179 


0.109 


0.226 


0.158 


67 


PR 


18 


0.252 


0.141 


0.215 


0.208 


68 


PR 


21 


0.084 


0.049 


0.097 


0.042 


69 


PR 


23 


0.115 


0.091 


0.112 


0.073 


70 


PR 


26 


0.183 


0.122 


0.172 


0.177 


71 


PR 


28 


0.174 


0.107 


0.139 


0.139 


72 


PK 


31 


0.171 


0.105 


0.154 


0.163 


73 


PR 


33 


0.126 


0.049 


0.191 


0.113 



17 18 19 20 21 22 23 24 

GKH 3 GKH 5 GKH 6 GtH 8 GKH 10 GKH 11 GKH 13 GIM 15 



1 .000 

0.173 1.000 

0.386 0.165 1.000 

0.067 0.119 0.095 1.000 

0.475 0.199 0.387 0.105 1.000 

0.370 0.122 0.227 0.060 0.316 1.000 

0.334 0.276 0.317 0.191 0.395 0.236 1.000 

0.278 0.204 0.238 0.169 0.324 0.185 0.391 1.000 

0.41 7 0.107 0.235 0.103 0. 31 6 0.235 0.178 0.2O9 

0.336 0.193 0.269 0.149 0.342 0.237 0.351 0.JO7 

0.307 0.144 0.230 0.103 0.231 0.198 0.291 0.256 

0.423 0.214 0.318 0.145. 0.380 0.243 0. 343 0.31 7 

0.362 0.243 0.272 0.159 0.343 0.257 0.405 0.35* 

0.298 0.039 0.224 0.082 0.258 0.180 0.175 0.169 

0.196 0.103 0. 11 3 0.093 0.166 0.128 0.170 0.127 

0.216 0.069 0.159 0.043 0.177 0.101 0.162 0.131 

0.293 0.060 0.162 0.033 0.197 0.137 0.184 0.129 

0.190 0.123 0.123 0.095 0.145 0.118 0.210 0.153 

0.171 0.090 0.090 0.042 0.142 0.075 0.168 0.127 

0.283 0.103 0.181 0.060 0.246 0.148 0.232 0.164 

0.282 0.098 0.183 0.044 0.178 0.133 0.210 0.165 

0.254 0.124 0.163 0.054 0.215 0.147 0.242 0.192 

0.193 0.081 0.109 0.068 0.158 0.036 0.129 0.137 

0.111 0.070 0.069 0.036 0.092 0.047 0.087 0.077 

0.218 0.080 0.124 0.036 0.168 0.124 0.177 0.143 

0.228 0.067 0.142 0.044 0.165 0.102 0.164 0.137 

0.122 0.020 0.080 0.03't 0.140 0.041 0.096 0.076 

0.213 0.052 0.107 0.058 0.202 0.125 0.176 0.112 

0.194 0.076 0.090 0.055 0.157 0.091 0.1S2 0.11J 

0.230 0.112 0.127 0.027 0.152 0.105 0.172 0.134 

0.369 0.143 0.270 0.095 0.337 0.227 0.351 0.280 

0.105 0.081 0.137 0.071 0.137 0.061 0.132 0.116 

0.404 0.141 0.252 0.093 0.354 0.245 0.320 0.259 

0.250 0.116 0.179 0.123 0.263 0.154 0.273 0.205 

0.199 0.062 0.146 0.024 0.153 0.112 0.176 0.128 

0.274 0.100 0.160 0.092 0.226 0.163 0.231 0.169 

0.324 0. 092 0.160 0.058 0.25 3 0.154 0.210 0.179 

0.263 0.158 0.173 0.075 0.197 0.141 0.229 0.169 

0.093 0.103 0.111 0.111 0.140 0.071 0.197 0.155 

0.020 -0.021 -0.002 0.022 -0.014 -0.004 -0.045 -0.035 

0.330 0.107 0.242 0.078 0.232 0. 196 0.235 0.19(1 

0.167 0.097 0.132 0.049 0.181 0.102 0.184 0.157 

0.191 0.131 0.196 0.082 0.216 0.180 0.259 0.201 

0.354 0.085 0.196 0.067 0.243 0.194 0.187 0.167 

0.226 0.065 0.131 0.011 0.173 0.123 0.129 0.102 

0.347 0.094 0.205 0.063 0.261 0.190 0.241 0.222 

0.229 0.069 0.159 0.050 0.212 0.155 0.168 0.145 

0.171 0.058 0.131 0.020 0.135 0.085 0.121 0.071 

0.243 0.092 0. 165 0.024 0.21 7 0.155 0.1 78 0.1 19 

0.341 0.102 0.168 0.052 0.226 0.159 0.176 0.154 

0.359 0.126 0.245 0.046 0.269 0.201 0.278 0.219 

0.197 0.024 0.083 0.037 0.107 0.085 0.082 0.055 

0.155 0.043 0.115 0.052 0.102 0.072 0.078 0.078 

0.241 0.134 0.179 0.071 0.231 0.136 0.216 0.17B 

0.231 0.116 0.188 0.040 0.205 0.106 0.212 0.159 

0.257 0.100 0.167 0.049 0.22 5 0.123 0.250 0.197 

0.212 0.062 0.118 0.027 0.173 0.100 0.111 0.064 
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SS25 

0. 151 
0.076 
0.173 
0.H5 
0.219 
0.213 
0.15$ 
0. 113 
0.102 
0. 178 
0. 116 
0.047 
0.182 
0. 191) 
0. 183 
0.213 
0.038 
0. 186 
0.186 
0.078 
0. 176 
0.248 
0.178 
0.1*7 
0. 1*1 
0.181 
0. 266 
0. 178 
0.183 
0.222 
0.118 
0.180 
0.136 
0.130 
0.082 
0.123 
0.073 
0. 099 
0.106 
0.099 
0. 198 
0. 180 
0. 196 
0.178 
0.132 
0.138 
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0.11* 
0.037 
0.1*0 
0.037 
0.136 
0.131 
0.083 
0.067 
0.102 
0.110 
0.096 
0.063 
0.133 
0.101 
0.1*6 
0.1S6 
0.020 
0.106 
0.133 
0.061 
0.130 
0.191 
0.176 
0.113 
0.106 
0.136 
0.172 
0.100 
0.108 
0.131 
0.070 
0.118 
0.071 
0.107 
0.O93 
0.133 
0.037 
0.073 
0.082 
0.129 
0.12* 
0.107 
0.1** 
0.119 
0.119 
0.117 
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0.132 
0.3*7 
0.221 
0.146 
0.196 
0.133 
0.099 
0.1*0 
0.080 
0.173 
0.2*9 
0.026 
0.19* 
0.221 
0.162 
0.162 
0.018 
0.216 
0.1 J9 
0.127 
0.230 
0.209 
0.183 
0.063 
0.096 
0.182 
0.286 
0.213 
0.229 
0.183 
0.172 
0.226 
0.13* 
0.083 
0.083 
0.127 
0.123 
0.103 
0.121 
0.123 
0.2*1 
0.132 
0.2*0 
0.123 
0.18* 
0.192 
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0.1* 

0.09 

0.12 

0.12 

0.17 

0.17 

0.19 

0.07 

0.06 

0.1 3 

0.13 

0.03 

0.130 

0.13 

0.1* 

0.16 

0.01 

0.22 

0.19 
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0.22 

0.2* 

0.1* 

0.1*0 

0.13 

0.13 

0.2 2 

0.13 

0.18 

0.20 

0.1* 

0.19 

0.1* 

0.09 

0.13 

0.1* 

0.1* 

0.12 

0.12 

0.13 

0.19 

0.160 

0.17 

0.130 

0.16 

0.1 3 
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0.26 

0.10 

0.3* 

0.270 

0.32 

0.28 

0.2* 

0.19 

0.100 

0.28 

0.33 

0.03 

0.3* 

0.23 

0.26 

0.27 

0.06 

0.33 

0.230 

0.1* 

0.3* 

0.39 

0.23 

0.17 

0.208 

0.29 

0.38 

0.28 

0.31 

0.36 

0.2* 
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0.222 
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1.000 
0.132 
0.O42 
0.144 
0.207 
0.102 
0.114 
0.148 
0.148 
0.297 
0.094 
0.33S 
0.220 
0.133 
0.198 
0.200 
0.241 
0.096 
-0.024 
0.247 
0.143 
0.187 
0.231 
0.148 
0.233 
0.144 
0.142 
0.179 
0.203 
0.317 
0.103 
0.039 
0.194 
0.214 
0.194 
0.103 
0.147 
0.072 
0.242 
0.178 
0.280 
0.234 
0.204 
0.121 
0.161 
0.202 
0.207 
0.061 
0.214 
0.244 
0.228 
0.207 
0.031 
0.264 
0.212 



1 .000 
0.200 
0.128 
0.134 
0.104 
0.0S9 
0.091 
0.04S 
0.133 
0.043 
0.179 
0.118 
0.097 
0.104 
0.102 
0.137 
0.044 
-0.021 
0.1S7 
0.072 
0.140 
0.089 
0.113 
0.179 
0.132 
0.093 
0.147 
0.131 
0.190 
0.063 
O.074 
0.131 
0.143 
0.132 
0.0S8 
0.117 
0.033 
0.218 
0.114 
0.179 
0.189 
0.139 
0.074 
0.092 
0.144 
0.174 
0.030 
0.138 
0.131 
0.124 
0.144 
0.033 
0.158 
0.165 



1.000 
0.043 
0.037 
0.071 
0.039 
0.071 
0.040 
0.07S 
0.042 
0.117 
0.039 
0.043 
0.071 
0.044 
0.047 
0.024 
-0.024 
0.043 
0.070 
0.084 
O.0S3 
0.038 
0.110 
0.030 
0.078 
0.033 
0.042 
0.072 
-0.003 
0.047 
0.097 
0.090 
0.094 
0.043 
0.040 
0.03S 
0.042 
0.034 
0.099 
0.108 
0.100 
0.039 
0.044 
0.094 
0.068 
0.014 
0.039 
0.107 
0.031 
0.104 
-0.004 
0.097 
0. 107 
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0.199 
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0.213 
0.041 
0.241 
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0.111 
0.173 
0.133 
0.148 
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0.132 
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0.149 
0.126 
0.181 
0.147 
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0.144 
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0.210 
0.048 
0.081 
0.120 
0.139 
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0.090 
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0.074 
0.174 
0.143 
0.204 
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0.421 
0.27* 
0.189 
0.316 
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0.280 
0.143 
-0.049 
0.342 
0.227 
0.268 
0.231 
0.16S 
0.300 
0.1S4 
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0.191 
0.242 
0.360 
0.099 
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0.130 
0.224 
0.124 
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0.214 
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0.16S 
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0.061 
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0.094 
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0.065 
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0.071 
0.076 
O.031 
0.0O8 
O.035 
0.065 
0.078 
0.072 
0.047 
0.110 
0.125 



BEST COPY AVAILABLE 



t« 

»5 
96 
97 
98 
99 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
110 



I 

00 



37 
1.16 
0. ioi 

0.224 
0.289 
0.208 
0.115 
0. 188 
0.205 
0.308 
0.210 
0.196 
0.219 
0.138 
0.241 
0.142 
0.105 
0.100 
0.182 
0.126 
0.134 
0.124 
0.103 
0.231 
0.171 
0.168 
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0.178 
0.154 
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U.OIO 

0.260 
0.330 
0.233 
0.147 
0.178 
0.243 
0.315 
0.200 
0.215 
0.260 
0.147 
0.230 
0.126 
0.135 
0.149 
0.148 
0.113 
0.140 
0.104 
0.169 
0.2SS 
0.201 
0.249 
0.200 
0.160 
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0.080 
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0.079 
0.077 
0.044 
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0.065 
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0.081 
0.089 
0.093 
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0.070 



41 
1.26 

0.083 
0.203 
0.190 
0.156 
0.145 
0.145 
0.185 
0.224 
0.163 
0.217 
0.205 
0.146 
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0.129 
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0.127 
0.094 
0.127 
0.115 
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0.131 
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0.117 
0.098 
0.116 
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0.150 
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0.061 
0.124 
0.091 
0.125 
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0.064 
0.107 
0.175 
0.128 
0.139 
0.137 
0.140 
0.101 
0.080 
0.070 
0.079 
0.103 
0.094 
0.073 
0.096 
0.090 
0.115 
0.119 
0.126 
0.115 
0.162 
0.202 



44 
1.33 

0.083 
0.173 
0.131 
0.146 
0.089 
0.128 
0.159 
0.190 
0.172 
0.215 
0.193 
0.185 
0.189 
0.106 
0.063 
0.OS2 
0.093 
0.124 
0.084 
0.150 
0.088 
0.167 
0.127 
0.202 
0.125 
0.185 
0.259 
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SI 

0.022 

0.191 

0.204 

0.112 

0.135 

0.146 

0.150 

0.223 

0.157 

0.153 
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0.063 
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0.127 
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0.133 
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0.116 
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0.175 
0.121 
0.1S4 
0.130 
0.079 
O.loO 
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0.061 
0.091 
0.082 
0.088 
0.057 
0.034 
0.131 
0.132 
0.151 
0.115 
0.096 
0.079 
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0.337 
0.266 
0.207 
0.203 
0.274 
0.346 
0.242 
0.258 
0.275 
0.190 
0.2.18 
0.179 
0.173 
0.157 
0.181 
0. 146 
0.179 
0.169 
0.154 
0.317 
0.240 
0.280 
0.245 
0.245 
0.216 
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0.056 
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0.043 

0.062 
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0.074 

0.084 
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0.063 

0.085 
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0.146 


78 


CSL 


11 
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0.221 
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0.216 


0.202 


0.233 
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0. 307 


0.223 
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0.203 


0.161 


0.192 
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0. 248 
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0.121 


0.167 


0.152 
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0.069 


0.135 


0.110 


82 


CSL 
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0.09a 


0.045 


0.096 


0.063 


0.116 


83 


CSL 
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0. 259 


0.176 


0.132 


0.142 


0.153 


0.144 


84 


CSL 


26 


0. 314 


0.1 74 


0.154 


0.172 


0.162 


0.179 


85 


CSL 


28 


0.055 


0.037 


0.012 


0.048 


0.040 


0.026 


86 


CSL 


31 


0. 300 


0.190 


0.185 


0.144 


0.144 


0.142 


87 


CSL 


33 


0. 339 


0.191 


0.143 


0.192 


0.148 


0.181 


88 


CSL 


36 


0.312 


0.216 


0.150 


0.207 


0.182 


0.165 


89 


CSL 


38 


0. 307 


0.194 


0.130 


0.199 


0.1d4 
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1.000 
0.007 
0.121 
0.074 
0.162 
0.081 
0.012 
0.071 
0.075 
0.003 
0.074 
0.077 
0.086 
0.026 
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